Businesses like banks that provide service have to worry about the problem of Churn i.e. customers leaving and joining another service provider. It is important to understand which aspects of the service influence a customer's decision in this regard. Management can concentrate efforts on the improvement of service, keeping in mind these priorities.
Given a Bank customer, build a neural network-based classifier that can determine whether they will leave or not in the next 6 months.
The case study is from an open-source dataset from Kaggle. The dataset contains 10,000 sample points with 14 distinct features such as CustomerId, CreditScore, Geography, Gender, Age, Tenure, Balance, etc.
CustomerId: Unique ID which is assigned to each customer
Surname: Last name of the customer
CreditScore: It defines the credit history of the customer.
Geography: A customer’s location
Gender: It defines the Gender of the customer
Age: Age of the customer
Tenure: Number of years for which the customer has been with the bank
NumOfProducts: It refers to the number of products that a customer has purchased through the bank.
Balance: Account balance
HasCrCard: It is a categorical variable that decides whether the customer has a credit card or not.
EstimatedSalary: Estimated salary
isActiveMember: It is a categorical variable that decides whether the customer is an active member of the bank or not ( Active member in the sense, using bank products regularly, making transactions, etc )
Exited: It is a categorical variable that decides whether the customer left the bank within six months or not. It can take two values
0=No ( Customer did not leave the bank )
1=Yes ( Customer left the bank )
Loading the required libraries
# Import the libraries needed. If not installed already, install them first using !pip install <library name>
# Libraries to help with reading and manipulating data
import pandas as pd
import numpy as np
# libaries to help with data visualization
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
# Library to encode the variables
from sklearn.preprocessing import OneHotEncoder
# Library to split data
from sklearn.model_selection import train_test_split
# library to import different optimizers
from tensorflow.keras import optimizers
# Library to import different loss functions
from tensorflow.keras import losses
from tensorflow.keras.layers import Dense
# Library to avoid the warnings
import warnings
warnings.filterwarnings('ignore')
# importing keras library
from tensorflow import keras
# library to convert the target variables to numpy arrays
from tensorflow.keras.utils import to_categorical
# library to plot classification report
from sklearn.metrics import classification_report
from sklearn.preprocessing import StandardScaler
import random
import tensorflow as tf
# library to import to standardize the data
from sklearn.preprocessing import MinMaxScaler
# importing different functions to build models
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation
# To get classification report
from sklearn.metrics import classification_report
# To plot confusion matrix
from sklearn.metrics import confusion_matrix
from tensorflow.keras import backend
from sklearn.metrics import (
f1_score,
accuracy_score,
recall_score,
precision_score,
confusion_matrix,
roc_auc_score,
plot_confusion_matrix,
precision_recall_curve,
roc_curve,
make_scorer,
)
from pandas.core.algorithms import value_counts
from imblearn.over_sampling import SMOTE
from imblearn.under_sampling import RandomUnderSampler
Mounting the Drive
# Connect with the Google drive so that it can read the data file.
from google.colab import drive
drive.mount('/content/drive/')
# Read the data file and take a sneek peak
data = pd.read_csv("/content/drive/My Drive/BankChurn.csv")
data.head()
Drive already mounted at /content/drive/; to attempt to forcibly remount, call drive.mount("/content/drive/", force_remount=True).
| RowNumber | CustomerId | Surname | CreditScore | Geography | Gender | Age | Tenure | Balance | NumOfProducts | HasCrCard | IsActiveMember | EstimatedSalary | Exited | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 15634602 | Hargrave | 619 | France | Female | 42 | 2 | 0.00 | 1 | 1 | 1 | 101348.88 | 1 |
| 1 | 2 | 15647311 | Hill | 608 | Spain | Female | 41 | 1 | 83807.86 | 1 | 0 | 1 | 112542.58 | 0 |
| 2 | 3 | 15619304 | Onio | 502 | France | Female | 42 | 8 | 159660.80 | 3 | 1 | 0 | 113931.57 | 1 |
| 3 | 4 | 15701354 | Boni | 699 | France | Female | 39 | 1 | 0.00 | 2 | 0 | 0 | 93826.63 | 0 |
| 4 | 5 | 15737888 | Mitchell | 850 | Spain | Female | 43 | 2 | 125510.82 | 1 | 1 | 1 | 79084.10 | 0 |
# Let's look at the basic row/column count
data.shape
(10000, 14)
# Let's look at basic non-null counts and data types
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 10000 entries, 0 to 9999 Data columns (total 14 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 RowNumber 10000 non-null int64 1 CustomerId 10000 non-null int64 2 Surname 10000 non-null object 3 CreditScore 10000 non-null int64 4 Geography 10000 non-null object 5 Gender 10000 non-null object 6 Age 10000 non-null int64 7 Tenure 10000 non-null int64 8 Balance 10000 non-null float64 9 NumOfProducts 10000 non-null int64 10 HasCrCard 10000 non-null int64 11 IsActiveMember 10000 non-null int64 12 EstimatedSalary 10000 non-null float64 13 Exited 10000 non-null int64 dtypes: float64(2), int64(9), object(3) memory usage: 1.1+ MB
# Let's look at the number of distinct values of the columns
data.nunique()
RowNumber 10000 CustomerId 10000 Surname 2932 CreditScore 460 Geography 3 Gender 2 Age 70 Tenure 11 Balance 6382 NumOfProducts 4 HasCrCard 2 IsActiveMember 2 EstimatedSalary 9999 Exited 2 dtype: int64
RowNumber and CustomerId are unique. They can be dropped. Surname is also not a useful field for model building. It can be dropped as well.
# Check how many fields have NA
data.isna().sum()
RowNumber 0 CustomerId 0 Surname 0 CreditScore 0 Geography 0 Gender 0 Age 0 Tenure 0 Balance 0 NumOfProducts 0 HasCrCard 0 IsActiveMember 0 EstimatedSalary 0 Exited 0 dtype: int64
# Check how many fields have nulls
data.isnull().sum()
RowNumber 0 CustomerId 0 Surname 0 CreditScore 0 Geography 0 Gender 0 Age 0 Tenure 0 Balance 0 NumOfProducts 0 HasCrCard 0 IsActiveMember 0 EstimatedSalary 0 Exited 0 dtype: int64
# Check for duplicate rows now
data.duplicated(keep=False).sum()
0
# Dropping columns that won't be part of the modeling
data = data.drop(['RowNumber', 'CustomerId','Surname'], axis = 1)
data.head()
| CreditScore | Geography | Gender | Age | Tenure | Balance | NumOfProducts | HasCrCard | IsActiveMember | EstimatedSalary | Exited | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 619 | France | Female | 42 | 2 | 0.00 | 1 | 1 | 1 | 101348.88 | 1 |
| 1 | 608 | Spain | Female | 41 | 1 | 83807.86 | 1 | 0 | 1 | 112542.58 | 0 |
| 2 | 502 | France | Female | 42 | 8 | 159660.80 | 3 | 1 | 0 | 113931.57 | 1 |
| 3 | 699 | France | Female | 39 | 1 | 0.00 | 2 | 0 | 0 | 93826.63 | 0 |
| 4 | 850 | Spain | Female | 43 | 2 | 125510.82 | 1 | 1 | 1 | 79084.10 | 0 |
# Let's confirm the changes
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 10000 entries, 0 to 9999 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 CreditScore 10000 non-null int64 1 Geography 10000 non-null object 2 Gender 10000 non-null object 3 Age 10000 non-null int64 4 Tenure 10000 non-null int64 5 Balance 10000 non-null float64 6 NumOfProducts 10000 non-null int64 7 HasCrCard 10000 non-null int64 8 IsActiveMember 10000 non-null int64 9 EstimatedSalary 10000 non-null float64 10 Exited 10000 non-null int64 dtypes: float64(2), int64(7), object(2) memory usage: 859.5+ KB
# Let's check the summary statistics of the data
data.describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| CreditScore | 10000.0 | 650.528800 | 96.653299 | 350.00 | 584.00 | 652.000 | 718.0000 | 850.00 |
| Age | 10000.0 | 38.921800 | 10.487806 | 18.00 | 32.00 | 37.000 | 44.0000 | 92.00 |
| Tenure | 10000.0 | 5.012800 | 2.892174 | 0.00 | 3.00 | 5.000 | 7.0000 | 10.00 |
| Balance | 10000.0 | 76485.889288 | 62397.405202 | 0.00 | 0.00 | 97198.540 | 127644.2400 | 250898.09 |
| NumOfProducts | 10000.0 | 1.530200 | 0.581654 | 1.00 | 1.00 | 1.000 | 2.0000 | 4.00 |
| HasCrCard | 10000.0 | 0.705500 | 0.455840 | 0.00 | 0.00 | 1.000 | 1.0000 | 1.00 |
| IsActiveMember | 10000.0 | 0.515100 | 0.499797 | 0.00 | 0.00 | 1.000 | 1.0000 | 1.00 |
| EstimatedSalary | 10000.0 | 100090.239881 | 57510.492818 | 11.58 | 51002.11 | 100193.915 | 149388.2475 | 199992.48 |
| Exited | 10000.0 | 0.203700 | 0.402769 | 0.00 | 0.00 | 0.000 | 0.0000 | 1.00 |
# EDA functions that will help us investigate the data
def histogram_boxplot(data, feature, figsize=(15, 10), kde=False, bins=None):
"""
Boxplot and histogram combined
data: dataframe
feature: dataframe column
figsize: size of figure (default (15,10))
kde: whether to show the density curve (default False)
bins: number of bins for histogram (default None)
"""
import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
import sys, os
#sys.path.append('/Users/Dhivya/Documents/Dhivya/Downloads')
f2, (ax_box2, ax_hist2) = plt.subplots(
nrows=2, # Number of rows of the subplot grid= 2
sharex=True, # x-axis will be shared among all subplots
gridspec_kw={"height_ratios": (0.25, 0.75)},
figsize=figsize,
) # creating the 2 subplots
sns.boxplot(
data=data, x=feature, ax=ax_box2, showmeans=True, color="violet"
) # boxplot will be created and a star will indicate the mean value of the column
sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins, palette="winter"
) if bins else sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2
) # For histogram
ax_hist2.axvline(
data[feature].mean(), color="green", linestyle="--"
) # Add mean to the histogram
ax_hist2.axvline(
data[feature].median(), color="black", linestyle="-"
) # Add median to the histogram
# function to create labeled barplots
def labeled_barplot(data, feature, perc=False, n=None):
"""
Barplot with percentage at the top
data: dataframe
feature: dataframe column
perc: whether to display percentages instead of count (default is False)
n: displays the top n category levels (default is None, i.e., display all levels)
"""
import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
import sys, os
#sys.path.append('/Users/Dhivya/Documents/Dhivya/Downloads')
total = len(data[feature]) # length of the column
count = data[feature].nunique()
if n is None:
plt.figure(figsize=(count + 2, 6))
else:
plt.figure(figsize=(n + 2, 6))
plt.xticks(rotation=90, fontsize=15)
ax = sns.countplot(
data=data,
x=feature,
palette="PuBu",
order=data[feature].value_counts(ascending=True).index[:n],
)
for p in ax.patches:
if perc == True:
label = "{:.1f}%".format(
100 * p.get_height() / total
) # percentage of each class of the category
else:
label = p.get_height() # count of each level of the category
x = p.get_x() + p.get_width() / 2 # width of the plot
y = p.get_height() # height of the plot
ax.annotate(
label,
(x, y),
ha="center",
va="center",
size=12,
xytext=(0, 5),
textcoords="offset points",
) # annotate the percentage
plt.show() # show the plot
def stacked_barplot(data, predictor, target):
"""
Print the category counts and plot a stacked bar chart
data: dataframe
predictor: independent variable
target: target variable
"""
import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
import sys, os
#sys.path.append('/Users/Dhivya/Documents/Dhivya/Downloads')
count = data[predictor].nunique()
sorter = data[target].value_counts().index[-1] # Sort by True
tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(
by=sorter, ascending=False
)
print(tab1)
print("-" * 50)
tab = pd.crosstab(data[predictor], data[target], margins=True,normalize="index").sort_values(
by=sorter, ascending=False
).round(4)*100
print(tab)
print("-" * 50)
tab.plot(kind="bar", stacked=True, figsize=(count + 5, 5))
#plt.legend(
#loc="lower left", frameon=True,
#)
plt.legend(loc="upper left", bbox_to_anchor=(1, 1))
plt.show()
def confusion_matrix_sklearn(model, predictors, target):
"""
To plot the confusion_matrix with percentages
model: classifier
predictors: independent variables
target: dependent variable
"""
y_pred = model.predict(predictors)
cm = confusion_matrix(target, y_pred)
labels = np.asarray(
[
["{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())]
for item in cm.flatten()
]
).reshape(2, 2)
plt.figure(figsize=(6, 4))
sns.heatmap(cm, annot=labels, fmt="")
plt.ylabel("True label")
plt.xlabel("Predicted label")
# The functions below display the model information and are taken from the code shared in the weekly sessions.
# defining a function to compute different metrics to check performance of a classification model built using sklearn
def model_performance_classification_sklearn_with_threshold(
model, predictors, target, threshold=0.5
):
"""
Function to compute different metrics, based on the threshold specified, to check classification model performance
model: classifier
predictors: independent variables
target: dependent variable
threshold: threshold for classifying the observation as class 1
"""
# predicting using the independent variables
pred_prob = model.predict_proba(predictors)[:, 1]
pred_thres = pred_prob > threshold
pred = np.round(pred_thres)
acc = accuracy_score(target, pred) # to compute Accuracy
recall = recall_score(target, pred) # to compute Recall
precision = precision_score(target, pred) # to compute Precision
f1 = f1_score(target, pred) # to compute F1-score
# creating a dataframe of metrics
df_perf = pd.DataFrame(
{
"Accuracy": acc,
"Recall": recall,
"Precision": precision,
"F1": f1,
},
index=[0],
)
return df_perf
Geography
labeled_barplot(data, "Geography", perc=True)
Gender
labeled_barplot(data, "Gender", perc=True)
# Call the above defined utility function for each numerical column
for col in data.select_dtypes(include=['int64','float64']).columns:
histogram_boxplot(data, col)
CreditScore - This has a normal distribution. There are some outliers on the low end. There is also a spike in customer count with the maximum credit score. Age - This has a somewhat right skewed distribution with many outliers on the high value end.Tenure - It is a fairly uniformly distributed.Balance - There are many zero balance customers. Barring this, it has a normal distribution.NumOfProducts - Most customers have one or two products. A much smaller number of customers have 3 or 4 products.HasCrCard - 70% customers have a credit card and 30% do not.IsActiveMember - this is even spilt between 0 and 1.EstimatedSalary - this is uniformly distributed between 11 and 199K. There isn't skew to justify doing a log function on this attribute.Exited - This is the target variable and 80% have a 0 and 20% have the value of 1. So this is an unbalanced distribution. So we may employ under-sampling and/or over-sampling to handle this.# Let's look at the correlation values of numerical attributes.
plt.figure(figsize=(15, 7))
sns.heatmap(
data.corr(), annot=True, vmin=-1, vmax=1, fmt=".2f", cmap="Spectral"
)
plt.show()
# Let's look at the distribution of numerical attributes showing a breakdown by the values of the categorical attributes.
ccol = data.select_dtypes(include=['object']).columns
ncol = data.select_dtypes(include=['int64','float64']).columns
for c1 in ccol:
for c2 in ncol:
plt.figure(figsize=(15,8))
a=sns.boxplot(x=c2,y=c1,data=data,orient="h")
# Let's look at the distribution of the numerical attributes for those attributes that have 0/1 values.
ccol = ['HasCrCard','IsActiveMember','Exited']
ncol = ['CreditScore','Age','Tenure','Balance','NumOfProducts','EstimatedSalary']
for c1 in ccol:
for c2 in ncol:
plt.figure(figsize=(15,8))
a=sns.boxplot(x=c2,y=c1,data=data,orient="h")
# Let's look at the stacked bar plots to continue the bivariate analysis
hcol = 'Exited'
vcols = ['Geography','Gender','Tenure','NumOfProducts','HasCrCard','IsActiveMember']
for vcol in vcols:
stacked_barplot(data,vcol,hcol)
Exited 0 1 All Geography All 7963 2037 10000 Germany 1695 814 2509 France 4204 810 5014 Spain 2064 413 2477 -------------------------------------------------- Exited 0 1 Geography Germany 67.56 32.44 All 79.63 20.37 Spain 83.33 16.67 France 83.85 16.15 --------------------------------------------------
Exited 0 1 All Gender All 7963 2037 10000 Female 3404 1139 4543 Male 4559 898 5457 -------------------------------------------------- Exited 0 1 Gender Female 74.93 25.07 All 79.63 20.37 Male 83.54 16.46 --------------------------------------------------
Exited 0 1 All Tenure All 7963 2037 10000 1 803 232 1035 3 796 213 1009 9 771 213 984 5 803 209 1012 4 786 203 989 2 847 201 1048 8 828 197 1025 6 771 196 967 7 851 177 1028 10 389 101 490 0 318 95 413 -------------------------------------------------- Exited 0 1 Tenure 0 77.00 23.00 1 77.58 22.42 9 78.35 21.65 3 78.89 21.11 5 79.35 20.65 10 79.39 20.61 4 79.47 20.53 All 79.63 20.37 6 79.73 20.27 8 80.78 19.22 2 80.82 19.18 7 82.78 17.22 --------------------------------------------------
Exited 0 1 All NumOfProducts All 7963 2037 10000 1 3675 1409 5084 2 4242 348 4590 3 46 220 266 4 0 60 60 -------------------------------------------------- Exited 0 1 NumOfProducts 4 0.00 100.00 3 17.29 82.71 1 72.29 27.71 All 79.63 20.37 2 92.42 7.58 --------------------------------------------------
Exited 0 1 All HasCrCard All 7963 2037 10000 1 5631 1424 7055 0 2332 613 2945 -------------------------------------------------- Exited 0 1 HasCrCard 0 79.19 20.81 All 79.63 20.37 1 79.82 20.18 --------------------------------------------------
Exited 0 1 All IsActiveMember All 7963 2037 10000 0 3547 1302 4849 1 4416 735 5151 -------------------------------------------------- Exited 0 1 IsActiveMember 0 73.15 26.85 All 79.63 20.37 1 85.73 14.27 --------------------------------------------------
# Let's get the 5 point summary and see how they vary for customers based on the target variable
data[data['Exited']==0].describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| CreditScore | 7963.0 | 651.853196 | 95.653837 | 405.00 | 585.00 | 653.00 | 718.000 | 850.00 |
| Age | 7963.0 | 37.408389 | 10.125363 | 18.00 | 31.00 | 36.00 | 41.000 | 92.00 |
| Tenure | 7963.0 | 5.033279 | 2.880658 | 0.00 | 3.00 | 5.00 | 7.000 | 10.00 |
| Balance | 7963.0 | 72745.296779 | 62848.040701 | 0.00 | 0.00 | 92072.68 | 126410.280 | 221532.80 |
| NumOfProducts | 7963.0 | 1.544267 | 0.509536 | 1.00 | 1.00 | 2.00 | 2.000 | 3.00 |
| HasCrCard | 7963.0 | 0.707146 | 0.455101 | 0.00 | 0.00 | 1.00 | 1.000 | 1.00 |
| IsActiveMember | 7963.0 | 0.554565 | 0.497045 | 0.00 | 0.00 | 1.00 | 1.000 | 1.00 |
| EstimatedSalary | 7963.0 | 99738.391772 | 57405.586966 | 90.07 | 50783.49 | 99645.04 | 148609.955 | 199992.48 |
| Exited | 7963.0 | 0.000000 | 0.000000 | 0.00 | 0.00 | 0.00 | 0.000 | 0.00 |
data[data['Exited']==1].describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| CreditScore | 2037.0 | 645.351497 | 100.321503 | 350.00 | 578.00 | 646.00 | 716.00 | 850.00 |
| Age | 2037.0 | 44.837997 | 9.761562 | 18.00 | 38.00 | 45.00 | 51.00 | 84.00 |
| Tenure | 2037.0 | 4.932744 | 2.936106 | 0.00 | 2.00 | 5.00 | 8.00 | 10.00 |
| Balance | 2037.0 | 91108.539337 | 58360.794816 | 0.00 | 38340.02 | 109349.29 | 131433.33 | 250898.09 |
| NumOfProducts | 2037.0 | 1.475209 | 0.801521 | 1.00 | 1.00 | 1.00 | 2.00 | 4.00 |
| HasCrCard | 2037.0 | 0.699067 | 0.458776 | 0.00 | 0.00 | 1.00 | 1.00 | 1.00 |
| IsActiveMember | 2037.0 | 0.360825 | 0.480358 | 0.00 | 0.00 | 0.00 | 1.00 | 1.00 |
| EstimatedSalary | 2037.0 | 101465.677531 | 57912.418071 | 11.58 | 51907.72 | 102460.84 | 152422.91 | 199808.10 |
| Exited | 2037.0 | 1.000000 | 0.000000 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
Exited) distribution varies by country, with the highest % of customers having exited in Germany and the lowest in France.# Let's continute the bivariate analysis using scatterplots.
sns.pairplot(data=data, diag_kind="kde")
plt.show()
# Let's extend the above by using a hue attribute.
ncols = ['CreditScore','Age','Tenure','Balance','NumOfProducts','EstimatedSalary']
hcols = ['Exited','HasCrCard','IsActiveMember']
for c1 in ncols:
for c2 in ncols:
if(c1>c2):
for hcol in hcols:
plt.figure(figsize = (15, 8))
sns.scatterplot(data = data,x = c1,y = c2,hue = hcol)
plt.show()
This dataset contains both numerical and categorical variables. We need to treat them before we pass them onto the neural network. We will perform the below pre-processing steps:
An important point to remember: Before we scale numerical variables, we would first split the dataset into train and test datasets and perform scaling separately. Otherwise, we would be leaking information from the test data to the train data and the resulting model might give a false sense of good performance. This is known as data leakage which we would like to avoid.
X = data.drop(['Exited'],axis=1)
y = data[['Exited']]
print(X.shape)
print(y.shape)
(10000, 10) (10000, 1)
# Let's do one hot encoding of categorical variables. Note that this step does not cause data leakage and hence can be done before splitting.
cat_data = ['Geography','Gender']
X = pd.get_dummies(X,columns=cat_data,drop_first= True)
Now, let's split the dataset into train and test datasets. To do that, we have already extracted all the independent variables and saved them into a variable X. And the target variable Exited are saved into a variable y. These two variables will be used to split the parent dataset into train and test datasets. The size of the dataset is small and the Keras implementation provides an argument for selecting some percentage of training data as validation data to check the accuracy of the model. Therefore, we will split the data into an 80:20 ratio.
# Splitting the dataset into the train (80%) and the test data (20%)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 5)
print(X.shape)
print(y.shape)
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
(10000, 11) (10000, 1) (8000, 11) (8000, 1) (2000, 11) (2000, 1)
Now, we will perform scaling on the numerical variables separately for train and test sets. We will perform fit and transform on the train data and then we will only perform transform on the test data.
## Scaling the data. Since y is categorical (0/1 values), we don't need to rescale y.
sc=StandardScaler()
temp = sc.fit(X_train)
X_train = temp.transform(X_train)
X_test = temp.transform(X_test)
print(X.shape)
print(y.shape)
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
(10000, 11) (10000, 1) (8000, 11) (8000, 1) (2000, 11) (2000, 1)
First, let's set the seed for random number generators in NumPy, Python, and TensorFlow to be able to reproduce the same results every time we run the code.
# Fixing the seed for random number generators
np.random.seed(5)
random.seed(5)
tf.random.set_seed(5)
In neural networks, there are so many hyper-parameters that we can play around with and tune the network to get the best results. Some of them are:
and so on.
Let's build a feed-forward neural network with 2 hidden layers and the output layer.
# We will be adding the layers sequentially
model_1 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_1.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_1.add(Dense(64, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of customer exiting
model_1.add(Dense(1, activation = 'sigmoid'))
Once we are done with the model architecture, we need to compile the model, where we need to provide the loss function that we want to optimize, the optimization algorithm, and the evaluation metric that we are interested in to evaluate the model.
Since this is a binary classification task, we will be minimizing the binary_crossentropy and we can choose one optimizer out of
This is a hyper-parameter. You can play around with these optimizers to check which one performs better with a particular data.
For now, let's try SGD optimizer with Recall as the metric and see the model's summary.
In the case of the Bank, both precision and recall are important. A false positive would lead to unnecessary promotional expenses and a false negative would lead to missed revenue opportunity. However, the cost of missed revenue is much bigger than the marginal promotional experience. Hence Recall is the chosen metric to optimize for.
# Compliling the model with binary crossentropy as loss, SGD as optimizer and accuracy as metrics
#model_1.compile(loss = 'binary_crossentropy', optimizer = 'SGD', metrics = ['accuracy'])
model_1.compile(loss = 'binary_crossentropy', optimizer = 'SGD', metrics=[tf.keras.metrics.Recall()])
# Printing the summary of the model
model_1.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 128) 1536
dense_1 (Dense) (None, 64) 8256
dense_2 (Dense) (None, 1) 65
=================================================================
Total params: 9,857
Trainable params: 9,857
Non-trainable params: 0
_________________________________________________________________
From the above summary, we can see that this architecture will train a total of 9,857 parameters, i.e., weights and biases in the network.
Let's now train the model using the below piece of code. We will keep 10% of the training data for validation.
history_1 = model_1.fit(X_train, y_train,validation_split = 0.1,epochs = 150,verbose = 2)
Epoch 1/150 225/225 - 1s - loss: 0.5418 - recall: 0.0657 - val_loss: 0.4740 - val_recall: 0.0000e+00 - 1s/epoch - 5ms/step Epoch 2/150 225/225 - 0s - loss: 0.4694 - recall: 0.0014 - val_loss: 0.4435 - val_recall: 0.0129 - 407ms/epoch - 2ms/step Epoch 3/150 225/225 - 0s - loss: 0.4456 - recall: 0.0460 - val_loss: 0.4248 - val_recall: 0.0903 - 370ms/epoch - 2ms/step Epoch 4/150 225/225 - 0s - loss: 0.4307 - recall: 0.1131 - val_loss: 0.4119 - val_recall: 0.1677 - 347ms/epoch - 2ms/step Epoch 5/150 225/225 - 0s - loss: 0.4206 - recall: 0.1774 - val_loss: 0.4021 - val_recall: 0.2258 - 384ms/epoch - 2ms/step Epoch 6/150 225/225 - 0s - loss: 0.4130 - recall: 0.2261 - val_loss: 0.3945 - val_recall: 0.2839 - 379ms/epoch - 2ms/step Epoch 7/150 225/225 - 0s - loss: 0.4066 - recall: 0.2729 - val_loss: 0.3868 - val_recall: 0.2839 - 342ms/epoch - 2ms/step Epoch 8/150 225/225 - 0s - loss: 0.4004 - recall: 0.2993 - val_loss: 0.3799 - val_recall: 0.2968 - 386ms/epoch - 2ms/step Epoch 9/150 225/225 - 0s - loss: 0.3946 - recall: 0.3128 - val_loss: 0.3739 - val_recall: 0.3677 - 379ms/epoch - 2ms/step Epoch 10/150 225/225 - 0s - loss: 0.3888 - recall: 0.3480 - val_loss: 0.3665 - val_recall: 0.3419 - 348ms/epoch - 2ms/step Epoch 11/150 225/225 - 0s - loss: 0.3834 - recall: 0.3453 - val_loss: 0.3605 - val_recall: 0.3871 - 338ms/epoch - 2ms/step Epoch 12/150 225/225 - 0s - loss: 0.3779 - recall: 0.3676 - val_loss: 0.3540 - val_recall: 0.3677 - 339ms/epoch - 2ms/step Epoch 13/150 225/225 - 0s - loss: 0.3727 - recall: 0.3859 - val_loss: 0.3476 - val_recall: 0.3742 - 355ms/epoch - 2ms/step Epoch 14/150 225/225 - 0s - loss: 0.3681 - recall: 0.3771 - val_loss: 0.3431 - val_recall: 0.4258 - 380ms/epoch - 2ms/step Epoch 15/150 225/225 - 0s - loss: 0.3637 - recall: 0.3988 - val_loss: 0.3372 - val_recall: 0.4065 - 350ms/epoch - 2ms/step Epoch 16/150 225/225 - 0s - loss: 0.3599 - recall: 0.4076 - val_loss: 0.3329 - val_recall: 0.3806 - 360ms/epoch - 2ms/step Epoch 17/150 225/225 - 0s - loss: 0.3566 - recall: 0.4218 - val_loss: 0.3290 - val_recall: 0.3806 - 380ms/epoch - 2ms/step Epoch 18/150 225/225 - 0s - loss: 0.3534 - recall: 0.4259 - val_loss: 0.3250 - val_recall: 0.4065 - 337ms/epoch - 1ms/step Epoch 19/150 225/225 - 0s - loss: 0.3508 - recall: 0.4367 - val_loss: 0.3223 - val_recall: 0.4387 - 352ms/epoch - 2ms/step Epoch 20/150 225/225 - 0s - loss: 0.3483 - recall: 0.4469 - val_loss: 0.3201 - val_recall: 0.4065 - 329ms/epoch - 1ms/step Epoch 21/150 225/225 - 0s - loss: 0.3464 - recall: 0.4462 - val_loss: 0.3178 - val_recall: 0.4258 - 334ms/epoch - 1ms/step Epoch 22/150 225/225 - 0s - loss: 0.3445 - recall: 0.4550 - val_loss: 0.3157 - val_recall: 0.4258 - 397ms/epoch - 2ms/step Epoch 23/150 225/225 - 0s - loss: 0.3428 - recall: 0.4543 - val_loss: 0.3149 - val_recall: 0.4710 - 391ms/epoch - 2ms/step Epoch 24/150 225/225 - 0s - loss: 0.3414 - recall: 0.4597 - val_loss: 0.3129 - val_recall: 0.4516 - 389ms/epoch - 2ms/step Epoch 25/150 225/225 - 0s - loss: 0.3401 - recall: 0.4617 - val_loss: 0.3115 - val_recall: 0.4387 - 379ms/epoch - 2ms/step Epoch 26/150 225/225 - 0s - loss: 0.3391 - recall: 0.4672 - val_loss: 0.3103 - val_recall: 0.4387 - 357ms/epoch - 2ms/step Epoch 27/150 225/225 - 0s - loss: 0.3379 - recall: 0.4685 - val_loss: 0.3092 - val_recall: 0.4516 - 377ms/epoch - 2ms/step Epoch 28/150 225/225 - 1s - loss: 0.3366 - recall: 0.4665 - val_loss: 0.3111 - val_recall: 0.5097 - 669ms/epoch - 3ms/step Epoch 29/150 225/225 - 1s - loss: 0.3359 - recall: 0.4766 - val_loss: 0.3078 - val_recall: 0.4645 - 750ms/epoch - 3ms/step Epoch 30/150 225/225 - 0s - loss: 0.3351 - recall: 0.4733 - val_loss: 0.3081 - val_recall: 0.4645 - 374ms/epoch - 2ms/step Epoch 31/150 225/225 - 0s - loss: 0.3340 - recall: 0.4780 - val_loss: 0.3075 - val_recall: 0.4645 - 395ms/epoch - 2ms/step Epoch 32/150 225/225 - 0s - loss: 0.3334 - recall: 0.4766 - val_loss: 0.3073 - val_recall: 0.4645 - 374ms/epoch - 2ms/step Epoch 33/150 225/225 - 0s - loss: 0.3326 - recall: 0.4780 - val_loss: 0.3067 - val_recall: 0.4581 - 385ms/epoch - 2ms/step Epoch 34/150 225/225 - 0s - loss: 0.3315 - recall: 0.4827 - val_loss: 0.3070 - val_recall: 0.4452 - 384ms/epoch - 2ms/step Epoch 35/150 225/225 - 0s - loss: 0.3311 - recall: 0.4787 - val_loss: 0.3067 - val_recall: 0.4581 - 354ms/epoch - 2ms/step Epoch 36/150 225/225 - 0s - loss: 0.3303 - recall: 0.4800 - val_loss: 0.3058 - val_recall: 0.4968 - 393ms/epoch - 2ms/step Epoch 37/150 225/225 - 0s - loss: 0.3298 - recall: 0.4800 - val_loss: 0.3054 - val_recall: 0.4645 - 383ms/epoch - 2ms/step Epoch 38/150 225/225 - 0s - loss: 0.3290 - recall: 0.4848 - val_loss: 0.3050 - val_recall: 0.4774 - 338ms/epoch - 2ms/step Epoch 39/150 225/225 - 0s - loss: 0.3286 - recall: 0.4807 - val_loss: 0.3056 - val_recall: 0.4968 - 359ms/epoch - 2ms/step Epoch 40/150 225/225 - 0s - loss: 0.3278 - recall: 0.4922 - val_loss: 0.3044 - val_recall: 0.4710 - 332ms/epoch - 1ms/step Epoch 41/150 225/225 - 0s - loss: 0.3273 - recall: 0.4834 - val_loss: 0.3050 - val_recall: 0.4581 - 375ms/epoch - 2ms/step Epoch 42/150 225/225 - 0s - loss: 0.3266 - recall: 0.4909 - val_loss: 0.3046 - val_recall: 0.4645 - 374ms/epoch - 2ms/step Epoch 43/150 225/225 - 0s - loss: 0.3259 - recall: 0.4848 - val_loss: 0.3048 - val_recall: 0.4968 - 335ms/epoch - 1ms/step Epoch 44/150 225/225 - 0s - loss: 0.3256 - recall: 0.4949 - val_loss: 0.3042 - val_recall: 0.4710 - 370ms/epoch - 2ms/step Epoch 45/150 225/225 - 0s - loss: 0.3251 - recall: 0.4936 - val_loss: 0.3050 - val_recall: 0.4710 - 347ms/epoch - 2ms/step Epoch 46/150 225/225 - 0s - loss: 0.3243 - recall: 0.4976 - val_loss: 0.3047 - val_recall: 0.4710 - 342ms/epoch - 2ms/step Epoch 47/150 225/225 - 0s - loss: 0.3237 - recall: 0.4922 - val_loss: 0.3041 - val_recall: 0.4903 - 388ms/epoch - 2ms/step Epoch 48/150 225/225 - 0s - loss: 0.3232 - recall: 0.4929 - val_loss: 0.3048 - val_recall: 0.4774 - 340ms/epoch - 2ms/step Epoch 49/150 225/225 - 0s - loss: 0.3226 - recall: 0.4922 - val_loss: 0.3046 - val_recall: 0.4774 - 381ms/epoch - 2ms/step Epoch 50/150 225/225 - 0s - loss: 0.3220 - recall: 0.4970 - val_loss: 0.3035 - val_recall: 0.4774 - 392ms/epoch - 2ms/step Epoch 51/150 225/225 - 0s - loss: 0.3215 - recall: 0.4963 - val_loss: 0.3053 - val_recall: 0.4903 - 349ms/epoch - 2ms/step Epoch 52/150 225/225 - 0s - loss: 0.3211 - recall: 0.4963 - val_loss: 0.3033 - val_recall: 0.4968 - 369ms/epoch - 2ms/step Epoch 53/150 225/225 - 0s - loss: 0.3207 - recall: 0.5071 - val_loss: 0.3033 - val_recall: 0.4516 - 374ms/epoch - 2ms/step Epoch 54/150 225/225 - 0s - loss: 0.3201 - recall: 0.4956 - val_loss: 0.3045 - val_recall: 0.4581 - 374ms/epoch - 2ms/step Epoch 55/150 225/225 - 0s - loss: 0.3198 - recall: 0.4976 - val_loss: 0.3035 - val_recall: 0.4839 - 380ms/epoch - 2ms/step Epoch 56/150 225/225 - 0s - loss: 0.3192 - recall: 0.5030 - val_loss: 0.3039 - val_recall: 0.4839 - 374ms/epoch - 2ms/step Epoch 57/150 225/225 - 0s - loss: 0.3186 - recall: 0.4983 - val_loss: 0.3028 - val_recall: 0.4903 - 342ms/epoch - 2ms/step Epoch 58/150 225/225 - 0s - loss: 0.3183 - recall: 0.5146 - val_loss: 0.3041 - val_recall: 0.4645 - 357ms/epoch - 2ms/step Epoch 59/150 225/225 - 0s - loss: 0.3180 - recall: 0.4997 - val_loss: 0.3029 - val_recall: 0.4774 - 383ms/epoch - 2ms/step Epoch 60/150 225/225 - 0s - loss: 0.3172 - recall: 0.5030 - val_loss: 0.3037 - val_recall: 0.4774 - 340ms/epoch - 2ms/step Epoch 61/150 225/225 - 0s - loss: 0.3166 - recall: 0.5152 - val_loss: 0.3043 - val_recall: 0.4452 - 351ms/epoch - 2ms/step Epoch 62/150 225/225 - 0s - loss: 0.3164 - recall: 0.5017 - val_loss: 0.3043 - val_recall: 0.4774 - 372ms/epoch - 2ms/step Epoch 63/150 225/225 - 0s - loss: 0.3157 - recall: 0.5037 - val_loss: 0.3027 - val_recall: 0.4645 - 365ms/epoch - 2ms/step Epoch 64/150 225/225 - 0s - loss: 0.3156 - recall: 0.5051 - val_loss: 0.3026 - val_recall: 0.4710 - 393ms/epoch - 2ms/step Epoch 65/150 225/225 - 0s - loss: 0.3148 - recall: 0.5024 - val_loss: 0.3041 - val_recall: 0.4839 - 350ms/epoch - 2ms/step Epoch 66/150 225/225 - 0s - loss: 0.3144 - recall: 0.5071 - val_loss: 0.3050 - val_recall: 0.4839 - 339ms/epoch - 2ms/step Epoch 67/150 225/225 - 0s - loss: 0.3142 - recall: 0.5146 - val_loss: 0.3032 - val_recall: 0.4839 - 355ms/epoch - 2ms/step Epoch 68/150 225/225 - 0s - loss: 0.3137 - recall: 0.5098 - val_loss: 0.3038 - val_recall: 0.4839 - 344ms/epoch - 2ms/step Epoch 69/150 225/225 - 0s - loss: 0.3132 - recall: 0.5105 - val_loss: 0.3036 - val_recall: 0.4968 - 369ms/epoch - 2ms/step Epoch 70/150 225/225 - 0s - loss: 0.3128 - recall: 0.5105 - val_loss: 0.3033 - val_recall: 0.4839 - 386ms/epoch - 2ms/step Epoch 71/150 225/225 - 0s - loss: 0.3124 - recall: 0.5105 - val_loss: 0.3033 - val_recall: 0.4968 - 330ms/epoch - 1ms/step Epoch 72/150 225/225 - 0s - loss: 0.3121 - recall: 0.5152 - val_loss: 0.3029 - val_recall: 0.4903 - 380ms/epoch - 2ms/step Epoch 73/150 225/225 - 0s - loss: 0.3117 - recall: 0.5146 - val_loss: 0.3030 - val_recall: 0.4839 - 353ms/epoch - 2ms/step Epoch 74/150 225/225 - 0s - loss: 0.3107 - recall: 0.5125 - val_loss: 0.3015 - val_recall: 0.4968 - 370ms/epoch - 2ms/step Epoch 75/150 225/225 - 0s - loss: 0.3105 - recall: 0.5179 - val_loss: 0.3030 - val_recall: 0.4968 - 416ms/epoch - 2ms/step Epoch 76/150 225/225 - 0s - loss: 0.3103 - recall: 0.5295 - val_loss: 0.3032 - val_recall: 0.4774 - 404ms/epoch - 2ms/step Epoch 77/150 225/225 - 0s - loss: 0.3101 - recall: 0.5166 - val_loss: 0.3057 - val_recall: 0.4839 - 340ms/epoch - 2ms/step Epoch 78/150 225/225 - 0s - loss: 0.3095 - recall: 0.5166 - val_loss: 0.3042 - val_recall: 0.4968 - 362ms/epoch - 2ms/step Epoch 79/150 225/225 - 0s - loss: 0.3087 - recall: 0.5173 - val_loss: 0.3047 - val_recall: 0.4968 - 341ms/epoch - 2ms/step Epoch 80/150 225/225 - 0s - loss: 0.3087 - recall: 0.5159 - val_loss: 0.3038 - val_recall: 0.4968 - 358ms/epoch - 2ms/step Epoch 81/150 225/225 - 0s - loss: 0.3081 - recall: 0.5240 - val_loss: 0.3028 - val_recall: 0.4903 - 355ms/epoch - 2ms/step Epoch 82/150 225/225 - 0s - loss: 0.3078 - recall: 0.5213 - val_loss: 0.3028 - val_recall: 0.4839 - 373ms/epoch - 2ms/step Epoch 83/150 225/225 - 0s - loss: 0.3076 - recall: 0.5206 - val_loss: 0.3036 - val_recall: 0.5032 - 373ms/epoch - 2ms/step Epoch 84/150 225/225 - 0s - loss: 0.3074 - recall: 0.5234 - val_loss: 0.3057 - val_recall: 0.4968 - 378ms/epoch - 2ms/step Epoch 85/150 225/225 - 0s - loss: 0.3066 - recall: 0.5206 - val_loss: 0.3025 - val_recall: 0.4903 - 367ms/epoch - 2ms/step Epoch 86/150 225/225 - 0s - loss: 0.3063 - recall: 0.5227 - val_loss: 0.3019 - val_recall: 0.4839 - 361ms/epoch - 2ms/step Epoch 87/150 225/225 - 0s - loss: 0.3062 - recall: 0.5227 - val_loss: 0.3044 - val_recall: 0.4903 - 346ms/epoch - 2ms/step Epoch 88/150 225/225 - 0s - loss: 0.3055 - recall: 0.5267 - val_loss: 0.3054 - val_recall: 0.5161 - 376ms/epoch - 2ms/step Epoch 89/150 225/225 - 0s - loss: 0.3050 - recall: 0.5254 - val_loss: 0.3038 - val_recall: 0.4903 - 382ms/epoch - 2ms/step Epoch 90/150 225/225 - 0s - loss: 0.3043 - recall: 0.5274 - val_loss: 0.3043 - val_recall: 0.4903 - 350ms/epoch - 2ms/step Epoch 91/150 225/225 - 0s - loss: 0.3042 - recall: 0.5267 - val_loss: 0.3055 - val_recall: 0.4839 - 341ms/epoch - 2ms/step Epoch 92/150 225/225 - 0s - loss: 0.3040 - recall: 0.5281 - val_loss: 0.3042 - val_recall: 0.4968 - 375ms/epoch - 2ms/step Epoch 93/150 225/225 - 0s - loss: 0.3035 - recall: 0.5308 - val_loss: 0.3078 - val_recall: 0.5032 - 366ms/epoch - 2ms/step Epoch 94/150 225/225 - 0s - loss: 0.3029 - recall: 0.5274 - val_loss: 0.3062 - val_recall: 0.4839 - 333ms/epoch - 1ms/step Epoch 95/150 225/225 - 0s - loss: 0.3027 - recall: 0.5254 - val_loss: 0.3032 - val_recall: 0.5032 - 336ms/epoch - 1ms/step Epoch 96/150 225/225 - 0s - loss: 0.3018 - recall: 0.5281 - val_loss: 0.3051 - val_recall: 0.4839 - 378ms/epoch - 2ms/step Epoch 97/150 225/225 - 0s - loss: 0.3018 - recall: 0.5308 - val_loss: 0.3002 - val_recall: 0.4839 - 381ms/epoch - 2ms/step Epoch 98/150 225/225 - 0s - loss: 0.3015 - recall: 0.5376 - val_loss: 0.3043 - val_recall: 0.5032 - 344ms/epoch - 2ms/step Epoch 99/150 225/225 - 0s - loss: 0.3010 - recall: 0.5301 - val_loss: 0.3078 - val_recall: 0.5032 - 345ms/epoch - 2ms/step Epoch 100/150 225/225 - 0s - loss: 0.3007 - recall: 0.5362 - val_loss: 0.3043 - val_recall: 0.4968 - 353ms/epoch - 2ms/step Epoch 101/150 225/225 - 0s - loss: 0.3005 - recall: 0.5416 - val_loss: 0.3035 - val_recall: 0.4774 - 382ms/epoch - 2ms/step Epoch 102/150 225/225 - 0s - loss: 0.3002 - recall: 0.5369 - val_loss: 0.3042 - val_recall: 0.4774 - 385ms/epoch - 2ms/step Epoch 103/150 225/225 - 0s - loss: 0.2998 - recall: 0.5342 - val_loss: 0.3035 - val_recall: 0.4968 - 360ms/epoch - 2ms/step Epoch 104/150 225/225 - 0s - loss: 0.2992 - recall: 0.5328 - val_loss: 0.3026 - val_recall: 0.4839 - 374ms/epoch - 2ms/step Epoch 105/150 225/225 - 0s - loss: 0.2990 - recall: 0.5369 - val_loss: 0.3040 - val_recall: 0.4903 - 352ms/epoch - 2ms/step Epoch 106/150 225/225 - 0s - loss: 0.2986 - recall: 0.5328 - val_loss: 0.3058 - val_recall: 0.4968 - 345ms/epoch - 2ms/step Epoch 107/150 225/225 - 0s - loss: 0.2983 - recall: 0.5355 - val_loss: 0.3089 - val_recall: 0.4903 - 348ms/epoch - 2ms/step Epoch 108/150 225/225 - 0s - loss: 0.2978 - recall: 0.5288 - val_loss: 0.3078 - val_recall: 0.5419 - 351ms/epoch - 2ms/step Epoch 109/150 225/225 - 0s - loss: 0.2979 - recall: 0.5383 - val_loss: 0.3080 - val_recall: 0.5355 - 378ms/epoch - 2ms/step Epoch 110/150 225/225 - 0s - loss: 0.2972 - recall: 0.5410 - val_loss: 0.3045 - val_recall: 0.4839 - 367ms/epoch - 2ms/step Epoch 111/150 225/225 - 0s - loss: 0.2970 - recall: 0.5308 - val_loss: 0.3072 - val_recall: 0.4968 - 407ms/epoch - 2ms/step Epoch 112/150 225/225 - 0s - loss: 0.2964 - recall: 0.5410 - val_loss: 0.3081 - val_recall: 0.5161 - 340ms/epoch - 2ms/step Epoch 113/150 225/225 - 0s - loss: 0.2962 - recall: 0.5430 - val_loss: 0.3073 - val_recall: 0.4710 - 381ms/epoch - 2ms/step Epoch 114/150 225/225 - 0s - loss: 0.2951 - recall: 0.5349 - val_loss: 0.3064 - val_recall: 0.4968 - 380ms/epoch - 2ms/step Epoch 115/150 225/225 - 0s - loss: 0.2952 - recall: 0.5437 - val_loss: 0.3053 - val_recall: 0.5161 - 372ms/epoch - 2ms/step Epoch 116/150 225/225 - 0s - loss: 0.2944 - recall: 0.5416 - val_loss: 0.3119 - val_recall: 0.5548 - 385ms/epoch - 2ms/step Epoch 117/150 225/225 - 0s - loss: 0.2946 - recall: 0.5464 - val_loss: 0.3063 - val_recall: 0.5032 - 345ms/epoch - 2ms/step Epoch 118/150 225/225 - 0s - loss: 0.2946 - recall: 0.5450 - val_loss: 0.3058 - val_recall: 0.5032 - 347ms/epoch - 2ms/step Epoch 119/150 225/225 - 0s - loss: 0.2937 - recall: 0.5355 - val_loss: 0.3079 - val_recall: 0.5161 - 375ms/epoch - 2ms/step Epoch 120/150 225/225 - 0s - loss: 0.2934 - recall: 0.5416 - val_loss: 0.3117 - val_recall: 0.5161 - 348ms/epoch - 2ms/step Epoch 121/150 225/225 - 0s - loss: 0.2931 - recall: 0.5511 - val_loss: 0.3074 - val_recall: 0.5097 - 372ms/epoch - 2ms/step Epoch 122/150 225/225 - 0s - loss: 0.2927 - recall: 0.5484 - val_loss: 0.3108 - val_recall: 0.4903 - 372ms/epoch - 2ms/step Epoch 123/150 225/225 - 0s - loss: 0.2921 - recall: 0.5450 - val_loss: 0.3082 - val_recall: 0.5097 - 335ms/epoch - 1ms/step Epoch 124/150 225/225 - 0s - loss: 0.2922 - recall: 0.5450 - val_loss: 0.3062 - val_recall: 0.5290 - 374ms/epoch - 2ms/step Epoch 125/150 225/225 - 0s - loss: 0.2916 - recall: 0.5491 - val_loss: 0.3077 - val_recall: 0.4903 - 345ms/epoch - 2ms/step Epoch 126/150 225/225 - 0s - loss: 0.2915 - recall: 0.5477 - val_loss: 0.3089 - val_recall: 0.5097 - 383ms/epoch - 2ms/step Epoch 127/150 225/225 - 0s - loss: 0.2907 - recall: 0.5498 - val_loss: 0.3039 - val_recall: 0.5226 - 344ms/epoch - 2ms/step Epoch 128/150 225/225 - 0s - loss: 0.2902 - recall: 0.5484 - val_loss: 0.3089 - val_recall: 0.5355 - 378ms/epoch - 2ms/step Epoch 129/150 225/225 - 0s - loss: 0.2901 - recall: 0.5545 - val_loss: 0.3095 - val_recall: 0.4968 - 341ms/epoch - 2ms/step Epoch 130/150 225/225 - 0s - loss: 0.2897 - recall: 0.5471 - val_loss: 0.3065 - val_recall: 0.4968 - 351ms/epoch - 2ms/step Epoch 131/150 225/225 - 0s - loss: 0.2894 - recall: 0.5477 - val_loss: 0.3094 - val_recall: 0.5290 - 341ms/epoch - 2ms/step Epoch 132/150 225/225 - 0s - loss: 0.2891 - recall: 0.5457 - val_loss: 0.3081 - val_recall: 0.4903 - 359ms/epoch - 2ms/step Epoch 133/150 225/225 - 0s - loss: 0.2887 - recall: 0.5504 - val_loss: 0.3105 - val_recall: 0.5097 - 375ms/epoch - 2ms/step Epoch 134/150 225/225 - 0s - loss: 0.2878 - recall: 0.5572 - val_loss: 0.3073 - val_recall: 0.5226 - 341ms/epoch - 2ms/step Epoch 135/150 225/225 - 0s - loss: 0.2878 - recall: 0.5484 - val_loss: 0.3096 - val_recall: 0.5290 - 377ms/epoch - 2ms/step Epoch 136/150 225/225 - 0s - loss: 0.2880 - recall: 0.5423 - val_loss: 0.3098 - val_recall: 0.5226 - 339ms/epoch - 2ms/step Epoch 137/150 225/225 - 0s - loss: 0.2870 - recall: 0.5592 - val_loss: 0.3075 - val_recall: 0.5290 - 333ms/epoch - 1ms/step Epoch 138/150 225/225 - 0s - loss: 0.2864 - recall: 0.5559 - val_loss: 0.3110 - val_recall: 0.5161 - 352ms/epoch - 2ms/step Epoch 139/150 225/225 - 0s - loss: 0.2865 - recall: 0.5491 - val_loss: 0.3099 - val_recall: 0.4903 - 360ms/epoch - 2ms/step Epoch 140/150 225/225 - 0s - loss: 0.2855 - recall: 0.5606 - val_loss: 0.3129 - val_recall: 0.5097 - 388ms/epoch - 2ms/step Epoch 141/150 225/225 - 0s - loss: 0.2854 - recall: 0.5559 - val_loss: 0.3128 - val_recall: 0.4903 - 354ms/epoch - 2ms/step Epoch 142/150 225/225 - 0s - loss: 0.2854 - recall: 0.5592 - val_loss: 0.3075 - val_recall: 0.5097 - 380ms/epoch - 2ms/step Epoch 143/150 225/225 - 0s - loss: 0.2845 - recall: 0.5552 - val_loss: 0.3084 - val_recall: 0.5097 - 344ms/epoch - 2ms/step Epoch 144/150 225/225 - 0s - loss: 0.2842 - recall: 0.5592 - val_loss: 0.3105 - val_recall: 0.5161 - 378ms/epoch - 2ms/step Epoch 145/150 225/225 - 0s - loss: 0.2839 - recall: 0.5680 - val_loss: 0.3084 - val_recall: 0.5097 - 377ms/epoch - 2ms/step Epoch 146/150 225/225 - 0s - loss: 0.2831 - recall: 0.5633 - val_loss: 0.3098 - val_recall: 0.5290 - 345ms/epoch - 2ms/step Epoch 147/150 225/225 - 0s - loss: 0.2834 - recall: 0.5613 - val_loss: 0.3096 - val_recall: 0.4903 - 390ms/epoch - 2ms/step Epoch 148/150 225/225 - 0s - loss: 0.2826 - recall: 0.5626 - val_loss: 0.3065 - val_recall: 0.4968 - 374ms/epoch - 2ms/step Epoch 149/150 225/225 - 0s - loss: 0.2823 - recall: 0.5680 - val_loss: 0.3086 - val_recall: 0.5161 - 383ms/epoch - 2ms/step Epoch 150/150 225/225 - 0s - loss: 0.2820 - recall: 0.5579 - val_loss: 0.3177 - val_recall: 0.5548 - 360ms/epoch - 2ms/step
plt.figure(figsize = (15, 8))
plt.plot(history_1.history['loss'])
plt.plot(history_1.history['val_loss'])
plt.title('Loss vs Epochs')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'lower right')
plt.show()
Observations:
Let's try to increase the model complexity by tuning some of the hyper-parameters mentioned earlier and check if we can improve the model performance. Out of all the options we have, let's try to change the number of hidden layers, the number of neurons in each hidden layer, and the optimizer from SGD to adam. Also, we have observed that validation loss became constant after some epochs, let's try less number of epochs which would also reduce the computation time.
First, we need to clear the previous model's history from the session. In Keras, we need special command to clear the model's history otherwise the previous model history remains in the backend. Also, let's fix the seed again after clearing the backend.
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(5)
random.seed(5)
tf.random.set_seed(5)
# We will be adding the layers sequentially
model_2 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_2.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_2.add(Dense(64, activation = 'relu'))
# Second hidden layer with 32 neurons and relu activation function
model_2.add(Dense(32, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of customer churning
model_2.add(Dense(1, activation = 'sigmoid'))
# Compliling the model with binary crossentropy as loss, Adam as optimizer and accuracy as metrics
model_2.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics=[tf.keras.metrics.Recall()])
# Printing the summary of the model
model_2.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 128) 1536
dense_1 (Dense) (None, 64) 8256
dense_2 (Dense) (None, 32) 2080
dense_3 (Dense) (None, 1) 33
=================================================================
Total params: 11,905
Trainable params: 11,905
Non-trainable params: 0
_________________________________________________________________
# Fitting the model on train data
history_2 = model_2.fit(X_train,y_train,validation_split = 0.1,epochs = 60,verbose = 2)
Epoch 1/60 225/225 - 1s - loss: 0.4152 - recall: 0.2877 - val_loss: 0.3384 - val_recall: 0.5677 - 1s/epoch - 6ms/step Epoch 2/60 225/225 - 0s - loss: 0.3549 - recall: 0.4685 - val_loss: 0.3193 - val_recall: 0.4387 - 451ms/epoch - 2ms/step Epoch 3/60 225/225 - 0s - loss: 0.3443 - recall: 0.4726 - val_loss: 0.3186 - val_recall: 0.5677 - 459ms/epoch - 2ms/step Epoch 4/60 225/225 - 0s - loss: 0.3368 - recall: 0.4997 - val_loss: 0.3183 - val_recall: 0.5097 - 478ms/epoch - 2ms/step Epoch 5/60 225/225 - 0s - loss: 0.3293 - recall: 0.4942 - val_loss: 0.3187 - val_recall: 0.5548 - 415ms/epoch - 2ms/step Epoch 6/60 225/225 - 0s - loss: 0.3255 - recall: 0.5058 - val_loss: 0.3232 - val_recall: 0.5032 - 427ms/epoch - 2ms/step Epoch 7/60 225/225 - 0s - loss: 0.3217 - recall: 0.5118 - val_loss: 0.3148 - val_recall: 0.4839 - 426ms/epoch - 2ms/step Epoch 8/60 225/225 - 0s - loss: 0.3175 - recall: 0.5173 - val_loss: 0.3115 - val_recall: 0.4710 - 434ms/epoch - 2ms/step Epoch 9/60 225/225 - 0s - loss: 0.3133 - recall: 0.5261 - val_loss: 0.3154 - val_recall: 0.4839 - 445ms/epoch - 2ms/step Epoch 10/60 225/225 - 0s - loss: 0.3094 - recall: 0.5410 - val_loss: 0.3226 - val_recall: 0.4258 - 477ms/epoch - 2ms/step Epoch 11/60 225/225 - 0s - loss: 0.3043 - recall: 0.5328 - val_loss: 0.3264 - val_recall: 0.5355 - 433ms/epoch - 2ms/step Epoch 12/60 225/225 - 0s - loss: 0.3031 - recall: 0.5416 - val_loss: 0.3246 - val_recall: 0.4968 - 443ms/epoch - 2ms/step Epoch 13/60 225/225 - 0s - loss: 0.2962 - recall: 0.5572 - val_loss: 0.3357 - val_recall: 0.5290 - 428ms/epoch - 2ms/step Epoch 14/60 225/225 - 0s - loss: 0.2926 - recall: 0.5443 - val_loss: 0.3336 - val_recall: 0.5548 - 455ms/epoch - 2ms/step Epoch 15/60 225/225 - 0s - loss: 0.2892 - recall: 0.5667 - val_loss: 0.3347 - val_recall: 0.4581 - 442ms/epoch - 2ms/step Epoch 16/60 225/225 - 0s - loss: 0.2857 - recall: 0.5579 - val_loss: 0.3289 - val_recall: 0.5032 - 424ms/epoch - 2ms/step Epoch 17/60 225/225 - 0s - loss: 0.2817 - recall: 0.5755 - val_loss: 0.3405 - val_recall: 0.4710 - 424ms/epoch - 2ms/step Epoch 18/60 225/225 - 0s - loss: 0.2763 - recall: 0.5768 - val_loss: 0.3367 - val_recall: 0.4516 - 442ms/epoch - 2ms/step Epoch 19/60 225/225 - 0s - loss: 0.2737 - recall: 0.5863 - val_loss: 0.3434 - val_recall: 0.5097 - 444ms/epoch - 2ms/step Epoch 20/60 225/225 - 0s - loss: 0.2707 - recall: 0.5985 - val_loss: 0.3476 - val_recall: 0.4194 - 466ms/epoch - 2ms/step Epoch 21/60 225/225 - 0s - loss: 0.2652 - recall: 0.6039 - val_loss: 0.3645 - val_recall: 0.4645 - 431ms/epoch - 2ms/step Epoch 22/60 225/225 - 0s - loss: 0.2597 - recall: 0.6141 - val_loss: 0.3599 - val_recall: 0.4516 - 464ms/epoch - 2ms/step Epoch 23/60 225/225 - 0s - loss: 0.2557 - recall: 0.6181 - val_loss: 0.3629 - val_recall: 0.5161 - 426ms/epoch - 2ms/step Epoch 24/60 225/225 - 0s - loss: 0.2531 - recall: 0.6263 - val_loss: 0.3531 - val_recall: 0.4968 - 458ms/epoch - 2ms/step Epoch 25/60 225/225 - 0s - loss: 0.2479 - recall: 0.6269 - val_loss: 0.3785 - val_recall: 0.5097 - 472ms/epoch - 2ms/step Epoch 26/60 225/225 - 0s - loss: 0.2472 - recall: 0.6412 - val_loss: 0.3917 - val_recall: 0.4258 - 458ms/epoch - 2ms/step Epoch 27/60 225/225 - 0s - loss: 0.2415 - recall: 0.6412 - val_loss: 0.3854 - val_recall: 0.4387 - 411ms/epoch - 2ms/step Epoch 28/60 225/225 - 0s - loss: 0.2378 - recall: 0.6534 - val_loss: 0.3907 - val_recall: 0.5613 - 415ms/epoch - 2ms/step Epoch 29/60 225/225 - 0s - loss: 0.2339 - recall: 0.6628 - val_loss: 0.3841 - val_recall: 0.4968 - 430ms/epoch - 2ms/step Epoch 30/60 225/225 - 0s - loss: 0.2272 - recall: 0.6811 - val_loss: 0.3979 - val_recall: 0.5677 - 458ms/epoch - 2ms/step Epoch 31/60 225/225 - 0s - loss: 0.2270 - recall: 0.6798 - val_loss: 0.4347 - val_recall: 0.4387 - 473ms/epoch - 2ms/step Epoch 32/60 225/225 - 0s - loss: 0.2232 - recall: 0.6798 - val_loss: 0.4169 - val_recall: 0.4968 - 457ms/epoch - 2ms/step Epoch 33/60 225/225 - 0s - loss: 0.2200 - recall: 0.6804 - val_loss: 0.4191 - val_recall: 0.4645 - 477ms/epoch - 2ms/step Epoch 34/60 225/225 - 0s - loss: 0.2138 - recall: 0.6892 - val_loss: 0.4264 - val_recall: 0.4387 - 435ms/epoch - 2ms/step Epoch 35/60 225/225 - 0s - loss: 0.2082 - recall: 0.7028 - val_loss: 0.4148 - val_recall: 0.5419 - 443ms/epoch - 2ms/step Epoch 36/60 225/225 - 0s - loss: 0.2055 - recall: 0.7028 - val_loss: 0.4288 - val_recall: 0.4903 - 437ms/epoch - 2ms/step Epoch 37/60 225/225 - 0s - loss: 0.2045 - recall: 0.7028 - val_loss: 0.4356 - val_recall: 0.5419 - 456ms/epoch - 2ms/step Epoch 38/60 225/225 - 0s - loss: 0.2032 - recall: 0.7116 - val_loss: 0.4506 - val_recall: 0.5355 - 490ms/epoch - 2ms/step Epoch 39/60 225/225 - 0s - loss: 0.1988 - recall: 0.7177 - val_loss: 0.4615 - val_recall: 0.6258 - 449ms/epoch - 2ms/step Epoch 40/60 225/225 - 0s - loss: 0.1940 - recall: 0.7271 - val_loss: 0.4628 - val_recall: 0.5613 - 442ms/epoch - 2ms/step Epoch 41/60 225/225 - 0s - loss: 0.1883 - recall: 0.7326 - val_loss: 0.4581 - val_recall: 0.5290 - 438ms/epoch - 2ms/step Epoch 42/60 225/225 - 0s - loss: 0.1860 - recall: 0.7285 - val_loss: 0.4757 - val_recall: 0.5419 - 420ms/epoch - 2ms/step Epoch 43/60 225/225 - 0s - loss: 0.1830 - recall: 0.7292 - val_loss: 0.5070 - val_recall: 0.4968 - 424ms/epoch - 2ms/step Epoch 44/60 225/225 - 0s - loss: 0.1808 - recall: 0.7400 - val_loss: 0.5081 - val_recall: 0.5419 - 421ms/epoch - 2ms/step Epoch 45/60 225/225 - 0s - loss: 0.1767 - recall: 0.7502 - val_loss: 0.5041 - val_recall: 0.5548 - 471ms/epoch - 2ms/step Epoch 46/60 225/225 - 0s - loss: 0.1685 - recall: 0.7603 - val_loss: 0.5014 - val_recall: 0.4774 - 451ms/epoch - 2ms/step Epoch 47/60 225/225 - 0s - loss: 0.1690 - recall: 0.7624 - val_loss: 0.5259 - val_recall: 0.5613 - 472ms/epoch - 2ms/step Epoch 48/60 225/225 - 0s - loss: 0.1618 - recall: 0.7739 - val_loss: 0.5456 - val_recall: 0.5161 - 454ms/epoch - 2ms/step Epoch 49/60 225/225 - 0s - loss: 0.1652 - recall: 0.7759 - val_loss: 0.5407 - val_recall: 0.5484 - 467ms/epoch - 2ms/step Epoch 50/60 225/225 - 0s - loss: 0.1600 - recall: 0.7759 - val_loss: 0.5765 - val_recall: 0.4645 - 415ms/epoch - 2ms/step Epoch 51/60 225/225 - 0s - loss: 0.1602 - recall: 0.7759 - val_loss: 0.5397 - val_recall: 0.5355 - 433ms/epoch - 2ms/step Epoch 52/60 225/225 - 0s - loss: 0.1525 - recall: 0.7915 - val_loss: 0.5639 - val_recall: 0.6000 - 454ms/epoch - 2ms/step Epoch 53/60 225/225 - 0s - loss: 0.1532 - recall: 0.7854 - val_loss: 0.5801 - val_recall: 0.5097 - 460ms/epoch - 2ms/step Epoch 54/60 225/225 - 0s - loss: 0.1474 - recall: 0.8003 - val_loss: 0.5996 - val_recall: 0.5419 - 458ms/epoch - 2ms/step Epoch 55/60 225/225 - 0s - loss: 0.1430 - recall: 0.7982 - val_loss: 0.5768 - val_recall: 0.5097 - 478ms/epoch - 2ms/step Epoch 56/60 225/225 - 0s - loss: 0.1435 - recall: 0.8037 - val_loss: 0.6127 - val_recall: 0.5032 - 428ms/epoch - 2ms/step Epoch 57/60 225/225 - 0s - loss: 0.1407 - recall: 0.8111 - val_loss: 0.5812 - val_recall: 0.5290 - 440ms/epoch - 2ms/step Epoch 58/60 225/225 - 0s - loss: 0.1387 - recall: 0.8165 - val_loss: 0.6014 - val_recall: 0.5355 - 475ms/epoch - 2ms/step Epoch 59/60 225/225 - 0s - loss: 0.1308 - recall: 0.8226 - val_loss: 0.6440 - val_recall: 0.5677 - 447ms/epoch - 2ms/step Epoch 60/60 225/225 - 0s - loss: 0.1319 - recall: 0.8267 - val_loss: 0.6193 - val_recall: 0.5097 - 425ms/epoch - 2ms/step
plt.figure(figsize = (15, 8))
plt.plot(history_2.history['loss'])
plt.plot(history_2.history['val_loss'])
plt.title('loss vs Epochs')
plt.ylabel('loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'lower right')
plt.show()
Observations:
Let's try to further tune some of the hyper-parameters and check if we can overcome the overfitting issue.
We will use learning_rate = 0.001 for the optimizer in the training process
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(5)
random.seed(5)
tf.random.set_seed(5)
# We will be adding the layers sequentially
model_3 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_3.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_3.add(Dense(64, activation = 'relu'))
# Second hidden layer with 32 neurons and relu activation function
model_3.add(Dense(32, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of students getting admitted into UCLA
model_3.add(Dense(1, activation = 'sigmoid'))
# Compliling the model with binary crossentropy as loss, Adam as optimizer with 0.001 as learning rate and accuracy as metrics
model_3.compile(loss = 'binary_crossentropy', optimizer = tf.keras.optimizers.Adam(learning_rate = 0.001), metrics=[tf.keras.metrics.Recall()])
# Printing the summary of the model
model_3.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 128) 1536
dense_1 (Dense) (None, 64) 8256
dense_2 (Dense) (None, 32) 2080
dense_3 (Dense) (None, 1) 33
=================================================================
Total params: 11,905
Trainable params: 11,905
Non-trainable params: 0
_________________________________________________________________
# Fitting the model on the train data
history_3 = model_3.fit(X_train,y_train,validation_split = 0.1,epochs = 50,verbose = 2)
Epoch 1/50 225/225 - 1s - loss: 0.4152 - recall: 0.2877 - val_loss: 0.3384 - val_recall: 0.5677 - 1s/epoch - 6ms/step Epoch 2/50 225/225 - 0s - loss: 0.3549 - recall: 0.4685 - val_loss: 0.3193 - val_recall: 0.4387 - 444ms/epoch - 2ms/step Epoch 3/50 225/225 - 0s - loss: 0.3443 - recall: 0.4726 - val_loss: 0.3186 - val_recall: 0.5677 - 454ms/epoch - 2ms/step Epoch 4/50 225/225 - 0s - loss: 0.3368 - recall: 0.4997 - val_loss: 0.3183 - val_recall: 0.5097 - 428ms/epoch - 2ms/step Epoch 5/50 225/225 - 0s - loss: 0.3293 - recall: 0.4942 - val_loss: 0.3187 - val_recall: 0.5548 - 461ms/epoch - 2ms/step Epoch 6/50 225/225 - 0s - loss: 0.3255 - recall: 0.5058 - val_loss: 0.3232 - val_recall: 0.5032 - 468ms/epoch - 2ms/step Epoch 7/50 225/225 - 0s - loss: 0.3217 - recall: 0.5118 - val_loss: 0.3148 - val_recall: 0.4839 - 472ms/epoch - 2ms/step Epoch 8/50 225/225 - 0s - loss: 0.3175 - recall: 0.5173 - val_loss: 0.3115 - val_recall: 0.4710 - 455ms/epoch - 2ms/step Epoch 9/50 225/225 - 0s - loss: 0.3133 - recall: 0.5261 - val_loss: 0.3154 - val_recall: 0.4839 - 469ms/epoch - 2ms/step Epoch 10/50 225/225 - 0s - loss: 0.3094 - recall: 0.5410 - val_loss: 0.3226 - val_recall: 0.4258 - 450ms/epoch - 2ms/step Epoch 11/50 225/225 - 0s - loss: 0.3043 - recall: 0.5328 - val_loss: 0.3264 - val_recall: 0.5355 - 474ms/epoch - 2ms/step Epoch 12/50 225/225 - 0s - loss: 0.3031 - recall: 0.5416 - val_loss: 0.3246 - val_recall: 0.4968 - 476ms/epoch - 2ms/step Epoch 13/50 225/225 - 0s - loss: 0.2962 - recall: 0.5572 - val_loss: 0.3357 - val_recall: 0.5290 - 421ms/epoch - 2ms/step Epoch 14/50 225/225 - 0s - loss: 0.2926 - recall: 0.5443 - val_loss: 0.3336 - val_recall: 0.5548 - 469ms/epoch - 2ms/step Epoch 15/50 225/225 - 0s - loss: 0.2892 - recall: 0.5667 - val_loss: 0.3347 - val_recall: 0.4581 - 426ms/epoch - 2ms/step Epoch 16/50 225/225 - 0s - loss: 0.2857 - recall: 0.5579 - val_loss: 0.3289 - val_recall: 0.5032 - 435ms/epoch - 2ms/step Epoch 17/50 225/225 - 0s - loss: 0.2817 - recall: 0.5755 - val_loss: 0.3405 - val_recall: 0.4710 - 457ms/epoch - 2ms/step Epoch 18/50 225/225 - 0s - loss: 0.2763 - recall: 0.5768 - val_loss: 0.3367 - val_recall: 0.4516 - 474ms/epoch - 2ms/step Epoch 19/50 225/225 - 0s - loss: 0.2737 - recall: 0.5863 - val_loss: 0.3434 - val_recall: 0.5097 - 436ms/epoch - 2ms/step Epoch 20/50 225/225 - 0s - loss: 0.2707 - recall: 0.5985 - val_loss: 0.3476 - val_recall: 0.4194 - 427ms/epoch - 2ms/step Epoch 21/50 225/225 - 0s - loss: 0.2652 - recall: 0.6039 - val_loss: 0.3645 - val_recall: 0.4645 - 428ms/epoch - 2ms/step Epoch 22/50 225/225 - 0s - loss: 0.2597 - recall: 0.6141 - val_loss: 0.3599 - val_recall: 0.4516 - 422ms/epoch - 2ms/step Epoch 23/50 225/225 - 0s - loss: 0.2557 - recall: 0.6181 - val_loss: 0.3629 - val_recall: 0.5161 - 446ms/epoch - 2ms/step Epoch 24/50 225/225 - 0s - loss: 0.2531 - recall: 0.6263 - val_loss: 0.3531 - val_recall: 0.4968 - 465ms/epoch - 2ms/step Epoch 25/50 225/225 - 0s - loss: 0.2479 - recall: 0.6269 - val_loss: 0.3785 - val_recall: 0.5097 - 471ms/epoch - 2ms/step Epoch 26/50 225/225 - 0s - loss: 0.2472 - recall: 0.6412 - val_loss: 0.3917 - val_recall: 0.4258 - 439ms/epoch - 2ms/step Epoch 27/50 225/225 - 0s - loss: 0.2415 - recall: 0.6412 - val_loss: 0.3854 - val_recall: 0.4387 - 456ms/epoch - 2ms/step Epoch 28/50 225/225 - 0s - loss: 0.2378 - recall: 0.6534 - val_loss: 0.3907 - val_recall: 0.5613 - 454ms/epoch - 2ms/step Epoch 29/50 225/225 - 0s - loss: 0.2339 - recall: 0.6628 - val_loss: 0.3841 - val_recall: 0.4968 - 431ms/epoch - 2ms/step Epoch 30/50 225/225 - 0s - loss: 0.2272 - recall: 0.6811 - val_loss: 0.3979 - val_recall: 0.5677 - 457ms/epoch - 2ms/step Epoch 31/50 225/225 - 0s - loss: 0.2270 - recall: 0.6798 - val_loss: 0.4347 - val_recall: 0.4387 - 418ms/epoch - 2ms/step Epoch 32/50 225/225 - 0s - loss: 0.2232 - recall: 0.6798 - val_loss: 0.4169 - val_recall: 0.4968 - 459ms/epoch - 2ms/step Epoch 33/50 225/225 - 0s - loss: 0.2200 - recall: 0.6804 - val_loss: 0.4191 - val_recall: 0.4645 - 437ms/epoch - 2ms/step Epoch 34/50 225/225 - 0s - loss: 0.2138 - recall: 0.6892 - val_loss: 0.4264 - val_recall: 0.4387 - 458ms/epoch - 2ms/step Epoch 35/50 225/225 - 0s - loss: 0.2082 - recall: 0.7028 - val_loss: 0.4148 - val_recall: 0.5419 - 458ms/epoch - 2ms/step Epoch 36/50 225/225 - 0s - loss: 0.2055 - recall: 0.7028 - val_loss: 0.4288 - val_recall: 0.4903 - 451ms/epoch - 2ms/step Epoch 37/50 225/225 - 0s - loss: 0.2045 - recall: 0.7028 - val_loss: 0.4356 - val_recall: 0.5419 - 452ms/epoch - 2ms/step Epoch 38/50 225/225 - 0s - loss: 0.2032 - recall: 0.7116 - val_loss: 0.4506 - val_recall: 0.5355 - 423ms/epoch - 2ms/step Epoch 39/50 225/225 - 0s - loss: 0.1988 - recall: 0.7177 - val_loss: 0.4615 - val_recall: 0.6258 - 463ms/epoch - 2ms/step Epoch 40/50 225/225 - 0s - loss: 0.1940 - recall: 0.7271 - val_loss: 0.4628 - val_recall: 0.5613 - 477ms/epoch - 2ms/step Epoch 41/50 225/225 - 0s - loss: 0.1883 - recall: 0.7326 - val_loss: 0.4581 - val_recall: 0.5290 - 419ms/epoch - 2ms/step Epoch 42/50 225/225 - 0s - loss: 0.1860 - recall: 0.7285 - val_loss: 0.4757 - val_recall: 0.5419 - 437ms/epoch - 2ms/step Epoch 43/50 225/225 - 0s - loss: 0.1830 - recall: 0.7292 - val_loss: 0.5070 - val_recall: 0.4968 - 465ms/epoch - 2ms/step Epoch 44/50 225/225 - 0s - loss: 0.1808 - recall: 0.7400 - val_loss: 0.5081 - val_recall: 0.5419 - 474ms/epoch - 2ms/step Epoch 45/50 225/225 - 0s - loss: 0.1767 - recall: 0.7502 - val_loss: 0.5041 - val_recall: 0.5548 - 467ms/epoch - 2ms/step Epoch 46/50 225/225 - 0s - loss: 0.1685 - recall: 0.7603 - val_loss: 0.5014 - val_recall: 0.4774 - 435ms/epoch - 2ms/step Epoch 47/50 225/225 - 0s - loss: 0.1690 - recall: 0.7624 - val_loss: 0.5259 - val_recall: 0.5613 - 455ms/epoch - 2ms/step Epoch 48/50 225/225 - 0s - loss: 0.1618 - recall: 0.7739 - val_loss: 0.5456 - val_recall: 0.5161 - 444ms/epoch - 2ms/step Epoch 49/50 225/225 - 0s - loss: 0.1652 - recall: 0.7759 - val_loss: 0.5407 - val_recall: 0.5484 - 421ms/epoch - 2ms/step Epoch 50/50 225/225 - 0s - loss: 0.1600 - recall: 0.7759 - val_loss: 0.5765 - val_recall: 0.4645 - 461ms/epoch - 2ms/step
plt.figure(figsize = (15, 8))
plt.plot(history_3.history['loss'])
plt.plot(history_3.history['val_loss'])
plt.title('loss vs Epochs')
plt.ylabel('loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'lower right')
plt.show()
# Since we cleared sessions, we need to recreate the model_1.
# We will be adding the layers sequentially
model_1 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_1.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_1.add(Dense(64, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of customer exiting
model_1.add(Dense(1, activation = 'sigmoid'))
# Compliling the model with binary crossentropy as loss, SGD as optimizer and accuracy as metrics
#model_1.compile(loss = 'binary_crossentropy', optimizer = 'SGD', metrics = ['accuracy'])
model_1.compile(loss = 'binary_crossentropy', optimizer = 'SGD', metrics=[tf.keras.metrics.Recall()])
history_1 = model_1.fit(X_train, y_train,validation_split = 0.1,epochs = 150,verbose = 0)
model_1.evaluate(X_test, y_test, verbose = 1)
test_pred = np.round(model_1.predict(X_test))
63/63 [==============================] - 0s 2ms/step - loss: 0.3679 - recall_1: 0.5012
The test accuracy is also coming out to be 84% which implies that our model is able to replicate the performance from the train and validation data on the test (unseen) data.
print(classification_report(y_test, test_pred))
cm = confusion_matrix(y_test, test_pred)
labels = np.asarray(
[
[
"{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())
] # flatten will reshape
for item in cm.flatten()
]
).reshape(2, 2)
plt.figure(figsize = (8, 5))
sns.heatmap(cm, annot = labels, fmt = '', xticklabels = ['Not Exited', 'Exited'], yticklabels = ['Not Exited', 'Exited'])
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()
precision recall f1-score support
0 0.88 0.92 0.90 1595
1 0.62 0.50 0.56 405
accuracy 0.84 2000
macro avg 0.75 0.71 0.73 2000
weighted avg 0.83 0.84 0.83 2000
# Let's plot the ROC curve (training dataset)
# roc_curve returns fpr : array, shape = [>2]
# Increasing false positive rates such that element i is the false
# positive rate of predictions with score >= thresholds[i].
# tpr : array, shape = [>2]
# Increasing true positive rates such that element i is the true
# positive rate of predictions with score >= thresholds[i].
# thresholds : array, shape = [n_thresholds]
# Decreasing thresholds on the decision function used to compute
# fpr and tpr.
logit_roc_auc_train = roc_auc_score(y_test, test_pred)
fpr, tpr, thresholds = roc_curve(y_test, test_pred)
plt.figure(figsize=(7, 5))
plt.plot(fpr, tpr, label="Logistic Regression (area = %0.2f)" % logit_roc_auc_train)
plt.plot([0, 1], [0, 1], "r--")
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("Receiver Operating Characteristic Curve")
plt.legend(loc="lower right")
plt.show()
# We will be adding the layers sequentially
model_4 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_4.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_4.add(Dense(64, activation = 'relu'))
# Second hidden layer with 32 neurons and relu activation function
model_4.add(Dense(32, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of customer churning
model_4.add(Dense(1, activation = 'sigmoid'))
# Compliling the model with binary crossentropy as loss, Adam as optimizer with 0.001 as learning rate and accuracy as metrics
model_4.compile(loss = 'binary_crossentropy', optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.001), metrics=[tf.keras.metrics.Recall()])
# Printing the summary of the model
model_4.summary()
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_7 (Dense) (None, 128) 1536
dense_8 (Dense) (None, 64) 8256
dense_9 (Dense) (None, 32) 2080
dense_10 (Dense) (None, 1) 33
=================================================================
Total params: 11,905
Trainable params: 11,905
Non-trainable params: 0
_________________________________________________________________
# Fitting the model on the train data
history_4 = model_4.fit(X_train,y_train,validation_split = 0.1,epochs = 50,verbose = 2)
Epoch 1/50 225/225 - 1s - loss: 0.4079 - recall_2: 0.2796 - val_loss: 0.3304 - val_recall_2: 0.4903 - 1s/epoch - 6ms/step Epoch 2/50 225/225 - 0s - loss: 0.3538 - recall_2: 0.4604 - val_loss: 0.3123 - val_recall_2: 0.4387 - 428ms/epoch - 2ms/step Epoch 3/50 225/225 - 0s - loss: 0.3447 - recall_2: 0.4794 - val_loss: 0.3083 - val_recall_2: 0.5419 - 408ms/epoch - 2ms/step Epoch 4/50 225/225 - 0s - loss: 0.3374 - recall_2: 0.4888 - val_loss: 0.3115 - val_recall_2: 0.4903 - 400ms/epoch - 2ms/step Epoch 5/50 225/225 - 0s - loss: 0.3332 - recall_2: 0.4888 - val_loss: 0.3137 - val_recall_2: 0.5742 - 441ms/epoch - 2ms/step Epoch 6/50 225/225 - 0s - loss: 0.3292 - recall_2: 0.4997 - val_loss: 0.3119 - val_recall_2: 0.5355 - 391ms/epoch - 2ms/step Epoch 7/50 225/225 - 0s - loss: 0.3259 - recall_2: 0.4976 - val_loss: 0.3066 - val_recall_2: 0.4710 - 424ms/epoch - 2ms/step Epoch 8/50 225/225 - 0s - loss: 0.3224 - recall_2: 0.5125 - val_loss: 0.3158 - val_recall_2: 0.4903 - 437ms/epoch - 2ms/step Epoch 9/50 225/225 - 0s - loss: 0.3192 - recall_2: 0.5146 - val_loss: 0.3121 - val_recall_2: 0.5226 - 433ms/epoch - 2ms/step Epoch 10/50 225/225 - 0s - loss: 0.3170 - recall_2: 0.5044 - val_loss: 0.3193 - val_recall_2: 0.4387 - 392ms/epoch - 2ms/step Epoch 11/50 225/225 - 0s - loss: 0.3151 - recall_2: 0.5058 - val_loss: 0.3170 - val_recall_2: 0.5097 - 376ms/epoch - 2ms/step Epoch 12/50 225/225 - 0s - loss: 0.3116 - recall_2: 0.5315 - val_loss: 0.3198 - val_recall_2: 0.5355 - 415ms/epoch - 2ms/step Epoch 13/50 225/225 - 0s - loss: 0.3091 - recall_2: 0.5328 - val_loss: 0.3155 - val_recall_2: 0.4645 - 431ms/epoch - 2ms/step Epoch 14/50 225/225 - 0s - loss: 0.3050 - recall_2: 0.5362 - val_loss: 0.3141 - val_recall_2: 0.5677 - 418ms/epoch - 2ms/step Epoch 15/50 225/225 - 0s - loss: 0.3014 - recall_2: 0.5423 - val_loss: 0.3174 - val_recall_2: 0.5161 - 424ms/epoch - 2ms/step Epoch 16/50 225/225 - 0s - loss: 0.3006 - recall_2: 0.5423 - val_loss: 0.3193 - val_recall_2: 0.5226 - 378ms/epoch - 2ms/step Epoch 17/50 225/225 - 0s - loss: 0.2975 - recall_2: 0.5531 - val_loss: 0.3341 - val_recall_2: 0.4323 - 419ms/epoch - 2ms/step Epoch 18/50 225/225 - 0s - loss: 0.2929 - recall_2: 0.5579 - val_loss: 0.3248 - val_recall_2: 0.5097 - 410ms/epoch - 2ms/step Epoch 19/50 225/225 - 0s - loss: 0.2928 - recall_2: 0.5559 - val_loss: 0.3314 - val_recall_2: 0.5677 - 420ms/epoch - 2ms/step Epoch 20/50 225/225 - 0s - loss: 0.2869 - recall_2: 0.5694 - val_loss: 0.3277 - val_recall_2: 0.4774 - 441ms/epoch - 2ms/step Epoch 21/50 225/225 - 0s - loss: 0.2855 - recall_2: 0.5674 - val_loss: 0.3420 - val_recall_2: 0.4258 - 366ms/epoch - 2ms/step Epoch 22/50 225/225 - 0s - loss: 0.2822 - recall_2: 0.5809 - val_loss: 0.3463 - val_recall_2: 0.4194 - 416ms/epoch - 2ms/step Epoch 23/50 225/225 - 0s - loss: 0.2809 - recall_2: 0.5741 - val_loss: 0.3545 - val_recall_2: 0.4387 - 397ms/epoch - 2ms/step Epoch 24/50 225/225 - 0s - loss: 0.2773 - recall_2: 0.5836 - val_loss: 0.3374 - val_recall_2: 0.4645 - 412ms/epoch - 2ms/step Epoch 25/50 225/225 - 0s - loss: 0.2737 - recall_2: 0.5965 - val_loss: 0.3620 - val_recall_2: 0.5806 - 421ms/epoch - 2ms/step Epoch 26/50 225/225 - 0s - loss: 0.2733 - recall_2: 0.6005 - val_loss: 0.3497 - val_recall_2: 0.5677 - 394ms/epoch - 2ms/step Epoch 27/50 225/225 - 0s - loss: 0.2689 - recall_2: 0.5965 - val_loss: 0.3484 - val_recall_2: 0.4774 - 414ms/epoch - 2ms/step Epoch 28/50 225/225 - 0s - loss: 0.2662 - recall_2: 0.6053 - val_loss: 0.3556 - val_recall_2: 0.6065 - 414ms/epoch - 2ms/step Epoch 29/50 225/225 - 0s - loss: 0.2609 - recall_2: 0.6168 - val_loss: 0.3674 - val_recall_2: 0.4903 - 416ms/epoch - 2ms/step Epoch 30/50 225/225 - 0s - loss: 0.2616 - recall_2: 0.6087 - val_loss: 0.3671 - val_recall_2: 0.5226 - 386ms/epoch - 2ms/step Epoch 31/50 225/225 - 0s - loss: 0.2589 - recall_2: 0.6229 - val_loss: 0.3630 - val_recall_2: 0.4774 - 376ms/epoch - 2ms/step Epoch 32/50 225/225 - 0s - loss: 0.2547 - recall_2: 0.6242 - val_loss: 0.3665 - val_recall_2: 0.5032 - 428ms/epoch - 2ms/step Epoch 33/50 225/225 - 0s - loss: 0.2533 - recall_2: 0.6344 - val_loss: 0.3618 - val_recall_2: 0.5226 - 408ms/epoch - 2ms/step Epoch 34/50 225/225 - 0s - loss: 0.2470 - recall_2: 0.6391 - val_loss: 0.3770 - val_recall_2: 0.5484 - 391ms/epoch - 2ms/step Epoch 35/50 225/225 - 0s - loss: 0.2451 - recall_2: 0.6432 - val_loss: 0.3585 - val_recall_2: 0.5419 - 423ms/epoch - 2ms/step Epoch 36/50 225/225 - 0s - loss: 0.2433 - recall_2: 0.6486 - val_loss: 0.3834 - val_recall_2: 0.6000 - 376ms/epoch - 2ms/step Epoch 37/50 225/225 - 0s - loss: 0.2438 - recall_2: 0.6479 - val_loss: 0.3875 - val_recall_2: 0.5806 - 409ms/epoch - 2ms/step Epoch 38/50 225/225 - 0s - loss: 0.2381 - recall_2: 0.6567 - val_loss: 0.3811 - val_recall_2: 0.5935 - 381ms/epoch - 2ms/step Epoch 39/50 225/225 - 0s - loss: 0.2355 - recall_2: 0.6547 - val_loss: 0.3852 - val_recall_2: 0.5935 - 387ms/epoch - 2ms/step Epoch 40/50 225/225 - 0s - loss: 0.2350 - recall_2: 0.6622 - val_loss: 0.3743 - val_recall_2: 0.5677 - 379ms/epoch - 2ms/step Epoch 41/50 225/225 - 0s - loss: 0.2313 - recall_2: 0.6669 - val_loss: 0.3958 - val_recall_2: 0.5161 - 404ms/epoch - 2ms/step Epoch 42/50 225/225 - 0s - loss: 0.2297 - recall_2: 0.6730 - val_loss: 0.3860 - val_recall_2: 0.5806 - 415ms/epoch - 2ms/step Epoch 43/50 225/225 - 0s - loss: 0.2268 - recall_2: 0.6710 - val_loss: 0.4154 - val_recall_2: 0.5742 - 444ms/epoch - 2ms/step Epoch 44/50 225/225 - 0s - loss: 0.2260 - recall_2: 0.6770 - val_loss: 0.3968 - val_recall_2: 0.5419 - 379ms/epoch - 2ms/step Epoch 45/50 225/225 - 0s - loss: 0.2222 - recall_2: 0.6750 - val_loss: 0.4069 - val_recall_2: 0.5226 - 417ms/epoch - 2ms/step Epoch 46/50 225/225 - 0s - loss: 0.2193 - recall_2: 0.6886 - val_loss: 0.3914 - val_recall_2: 0.5290 - 404ms/epoch - 2ms/step Epoch 47/50 225/225 - 0s - loss: 0.2170 - recall_2: 0.7028 - val_loss: 0.4049 - val_recall_2: 0.5677 - 410ms/epoch - 2ms/step Epoch 48/50 225/225 - 0s - loss: 0.2184 - recall_2: 0.7055 - val_loss: 0.4167 - val_recall_2: 0.5290 - 403ms/epoch - 2ms/step Epoch 49/50 225/225 - 0s - loss: 0.2121 - recall_2: 0.7068 - val_loss: 0.4372 - val_recall_2: 0.4774 - 403ms/epoch - 2ms/step Epoch 50/50 225/225 - 0s - loss: 0.2102 - recall_2: 0.7048 - val_loss: 0.4164 - val_recall_2: 0.5419 - 425ms/epoch - 2ms/step
plt.figure(figsize = (15, 8))
plt.plot(history_4.history['loss'])
plt.plot(history_4.history['val_loss'])
plt.title('loss vs Epochs')
plt.ylabel('loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'lower right')
plt.show()
# We will be adding the layers sequentially
model_5 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_5.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_5.add(Dense(64, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of customer exiting
model_5.add(Dense(1, activation = 'sigmoid'))
# Compliling the model with binary crossentropy as loss, Adam as optimizer with 0.001 as learning rate and accuracy as metrics
model_5.compile(loss = 'binary_crossentropy', optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.001), metrics=[tf.keras.metrics.Recall()])
# Printing the summary of the model
model_5.summary()
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_11 (Dense) (None, 128) 1536
dense_12 (Dense) (None, 64) 8256
dense_13 (Dense) (None, 1) 65
=================================================================
Total params: 9,857
Trainable params: 9,857
Non-trainable params: 0
_________________________________________________________________
# Fitting the model on the train data
history_5 = model_5.fit(X_train,y_train,validation_split = 0.1,epochs = 50,verbose = 2)
Epoch 1/50 225/225 - 1s - loss: 0.4284 - recall_3: 0.2221 - val_loss: 0.3680 - val_recall_3: 0.3871 - 1s/epoch - 6ms/step Epoch 2/50 225/225 - 0s - loss: 0.3742 - recall_3: 0.3859 - val_loss: 0.3234 - val_recall_3: 0.4323 - 429ms/epoch - 2ms/step Epoch 3/50 225/225 - 0s - loss: 0.3516 - recall_3: 0.4455 - val_loss: 0.3094 - val_recall_3: 0.5226 - 396ms/epoch - 2ms/step Epoch 4/50 225/225 - 0s - loss: 0.3415 - recall_3: 0.4766 - val_loss: 0.3106 - val_recall_3: 0.4645 - 358ms/epoch - 2ms/step Epoch 5/50 225/225 - 0s - loss: 0.3371 - recall_3: 0.4821 - val_loss: 0.3040 - val_recall_3: 0.5548 - 405ms/epoch - 2ms/step Epoch 6/50 225/225 - 0s - loss: 0.3319 - recall_3: 0.4895 - val_loss: 0.3106 - val_recall_3: 0.5097 - 380ms/epoch - 2ms/step Epoch 7/50 225/225 - 0s - loss: 0.3290 - recall_3: 0.5003 - val_loss: 0.2998 - val_recall_3: 0.4710 - 404ms/epoch - 2ms/step Epoch 8/50 225/225 - 0s - loss: 0.3258 - recall_3: 0.5024 - val_loss: 0.3065 - val_recall_3: 0.4581 - 411ms/epoch - 2ms/step Epoch 9/50 225/225 - 0s - loss: 0.3223 - recall_3: 0.5071 - val_loss: 0.3046 - val_recall_3: 0.4968 - 372ms/epoch - 2ms/step Epoch 10/50 225/225 - 0s - loss: 0.3198 - recall_3: 0.5132 - val_loss: 0.3095 - val_recall_3: 0.4129 - 407ms/epoch - 2ms/step Epoch 11/50 225/225 - 0s - loss: 0.3187 - recall_3: 0.4997 - val_loss: 0.3089 - val_recall_3: 0.4903 - 416ms/epoch - 2ms/step Epoch 12/50 225/225 - 0s - loss: 0.3155 - recall_3: 0.5193 - val_loss: 0.3132 - val_recall_3: 0.4774 - 409ms/epoch - 2ms/step Epoch 13/50 225/225 - 0s - loss: 0.3135 - recall_3: 0.5301 - val_loss: 0.3130 - val_recall_3: 0.4516 - 408ms/epoch - 2ms/step Epoch 14/50 225/225 - 0s - loss: 0.3099 - recall_3: 0.5159 - val_loss: 0.3101 - val_recall_3: 0.5226 - 423ms/epoch - 2ms/step Epoch 15/50 225/225 - 0s - loss: 0.3079 - recall_3: 0.5349 - val_loss: 0.3139 - val_recall_3: 0.4774 - 382ms/epoch - 2ms/step Epoch 16/50 225/225 - 0s - loss: 0.3065 - recall_3: 0.5403 - val_loss: 0.3119 - val_recall_3: 0.5097 - 362ms/epoch - 2ms/step Epoch 17/50 225/225 - 0s - loss: 0.3052 - recall_3: 0.5423 - val_loss: 0.3257 - val_recall_3: 0.4129 - 405ms/epoch - 2ms/step Epoch 18/50 225/225 - 1s - loss: 0.3024 - recall_3: 0.5437 - val_loss: 0.3154 - val_recall_3: 0.4774 - 947ms/epoch - 4ms/step Epoch 19/50 225/225 - 0s - loss: 0.3013 - recall_3: 0.5376 - val_loss: 0.3177 - val_recall_3: 0.5548 - 496ms/epoch - 2ms/step Epoch 20/50 225/225 - 0s - loss: 0.2983 - recall_3: 0.5484 - val_loss: 0.3223 - val_recall_3: 0.4258 - 397ms/epoch - 2ms/step Epoch 21/50 225/225 - 0s - loss: 0.2960 - recall_3: 0.5498 - val_loss: 0.3282 - val_recall_3: 0.4581 - 386ms/epoch - 2ms/step Epoch 22/50 225/225 - 0s - loss: 0.2937 - recall_3: 0.5599 - val_loss: 0.3338 - val_recall_3: 0.4258 - 369ms/epoch - 2ms/step Epoch 23/50 225/225 - 0s - loss: 0.2934 - recall_3: 0.5538 - val_loss: 0.3299 - val_recall_3: 0.4645 - 401ms/epoch - 2ms/step Epoch 24/50 225/225 - 0s - loss: 0.2898 - recall_3: 0.5572 - val_loss: 0.3225 - val_recall_3: 0.4581 - 381ms/epoch - 2ms/step Epoch 25/50 225/225 - 0s - loss: 0.2882 - recall_3: 0.5674 - val_loss: 0.3328 - val_recall_3: 0.4968 - 396ms/epoch - 2ms/step Epoch 26/50 225/225 - 0s - loss: 0.2881 - recall_3: 0.5647 - val_loss: 0.3237 - val_recall_3: 0.5097 - 378ms/epoch - 2ms/step Epoch 27/50 225/225 - 0s - loss: 0.2854 - recall_3: 0.5674 - val_loss: 0.3351 - val_recall_3: 0.4645 - 373ms/epoch - 2ms/step Epoch 28/50 225/225 - 0s - loss: 0.2829 - recall_3: 0.5768 - val_loss: 0.3316 - val_recall_3: 0.5871 - 364ms/epoch - 2ms/step Epoch 29/50 225/225 - 0s - loss: 0.2802 - recall_3: 0.5856 - val_loss: 0.3374 - val_recall_3: 0.4968 - 375ms/epoch - 2ms/step Epoch 30/50 225/225 - 0s - loss: 0.2806 - recall_3: 0.5850 - val_loss: 0.3346 - val_recall_3: 0.5548 - 360ms/epoch - 2ms/step Epoch 31/50 225/225 - 0s - loss: 0.2783 - recall_3: 0.5863 - val_loss: 0.3412 - val_recall_3: 0.4452 - 409ms/epoch - 2ms/step Epoch 32/50 225/225 - 0s - loss: 0.2763 - recall_3: 0.5836 - val_loss: 0.3309 - val_recall_3: 0.4968 - 356ms/epoch - 2ms/step Epoch 33/50 225/225 - 0s - loss: 0.2738 - recall_3: 0.5972 - val_loss: 0.3359 - val_recall_3: 0.4903 - 415ms/epoch - 2ms/step Epoch 34/50 225/225 - 0s - loss: 0.2714 - recall_3: 0.6012 - val_loss: 0.3427 - val_recall_3: 0.4645 - 375ms/epoch - 2ms/step Epoch 35/50 225/225 - 0s - loss: 0.2699 - recall_3: 0.5978 - val_loss: 0.3378 - val_recall_3: 0.4645 - 354ms/epoch - 2ms/step Epoch 36/50 225/225 - 0s - loss: 0.2694 - recall_3: 0.5917 - val_loss: 0.3399 - val_recall_3: 0.5613 - 374ms/epoch - 2ms/step Epoch 37/50 225/225 - 0s - loss: 0.2688 - recall_3: 0.5951 - val_loss: 0.3406 - val_recall_3: 0.5290 - 388ms/epoch - 2ms/step Epoch 38/50 225/225 - 0s - loss: 0.2648 - recall_3: 0.6019 - val_loss: 0.3416 - val_recall_3: 0.5742 - 389ms/epoch - 2ms/step Epoch 39/50 225/225 - 0s - loss: 0.2631 - recall_3: 0.6039 - val_loss: 0.3418 - val_recall_3: 0.5677 - 440ms/epoch - 2ms/step Epoch 40/50 225/225 - 0s - loss: 0.2626 - recall_3: 0.6100 - val_loss: 0.3365 - val_recall_3: 0.5548 - 362ms/epoch - 2ms/step Epoch 41/50 225/225 - 0s - loss: 0.2612 - recall_3: 0.6134 - val_loss: 0.3454 - val_recall_3: 0.5032 - 402ms/epoch - 2ms/step Epoch 42/50 225/225 - 0s - loss: 0.2596 - recall_3: 0.6134 - val_loss: 0.3415 - val_recall_3: 0.5548 - 407ms/epoch - 2ms/step Epoch 43/50 225/225 - 0s - loss: 0.2573 - recall_3: 0.6175 - val_loss: 0.3587 - val_recall_3: 0.5097 - 361ms/epoch - 2ms/step Epoch 44/50 225/225 - 0s - loss: 0.2563 - recall_3: 0.6161 - val_loss: 0.3480 - val_recall_3: 0.5032 - 412ms/epoch - 2ms/step Epoch 45/50 225/225 - 0s - loss: 0.2552 - recall_3: 0.6222 - val_loss: 0.3576 - val_recall_3: 0.5161 - 405ms/epoch - 2ms/step Epoch 46/50 225/225 - 0s - loss: 0.2522 - recall_3: 0.6439 - val_loss: 0.3518 - val_recall_3: 0.5161 - 382ms/epoch - 2ms/step Epoch 47/50 225/225 - 0s - loss: 0.2513 - recall_3: 0.6256 - val_loss: 0.3479 - val_recall_3: 0.5613 - 372ms/epoch - 2ms/step Epoch 48/50 225/225 - 0s - loss: 0.2492 - recall_3: 0.6344 - val_loss: 0.3665 - val_recall_3: 0.4710 - 408ms/epoch - 2ms/step Epoch 49/50 225/225 - 0s - loss: 0.2469 - recall_3: 0.6452 - val_loss: 0.3580 - val_recall_3: 0.4903 - 362ms/epoch - 2ms/step Epoch 50/50 225/225 - 0s - loss: 0.2449 - recall_3: 0.6357 - val_loss: 0.3525 - val_recall_3: 0.4839 - 404ms/epoch - 2ms/step
plt.figure(figsize = (15, 8))
plt.plot(history_5.history['loss'])
plt.plot(history_5.history['val_loss'])
plt.title('loss vs Epochs')
plt.ylabel('loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'lower right')
plt.show()
# We use smote to oversample as a way to handle the class imbalance.
print("Before Oversampling")
print(y_train.value_counts())
sm = SMOTE(
sampling_strategy=1, k_neighbors=5, random_state=1
) # Synthetic Minority Over Sampling Technique
X_train_over, y_train_over = sm.fit_resample(X_train, y_train)
print("After Oversampling")
print(y_train_over.value_counts())
print("After Oversampling, the shape of train_X: {}".format(X_train_over.shape))
print("After Oversampling, the shape of train_y: {} \n".format(y_train_over.shape))
Before Oversampling Exited 0 6368 1 1632 dtype: int64 After Oversampling Exited 0 6368 1 6368 dtype: int64 After Oversampling, the shape of train_X: (12736, 11) After Oversampling, the shape of train_y: (12736, 1)
# We will be adding the layers sequentially
model_6 = Sequential()
# First hidden layer with 128 neurons and relu activation function, the input_shape tuple denotes number of independent variables
model_6.add(Dense(128, activation = 'relu', input_shape = (11, )))
# Second hidden layer with 64 neurons and relu activation function
model_6.add(Dense(64, activation = 'relu'))
# Output layer with only one neuron and sigmoid as activation function which will give the probability of customer exiting
model_6.add(Dense(1, activation = 'sigmoid'))
# Compliling the model with binary crossentropy as loss, SGD as optimizer and accuracy as metrics
model_6.compile(loss = 'binary_crossentropy', optimizer = 'SGD', metrics=[tf.keras.metrics.Recall()])
history_6 = model_6.fit(X_train_over, y_train_over,validation_split = 0.1,epochs = 150,verbose = 2)
model_6.evaluate(X_test, y_test, verbose = 1)
test_pred = np.round(model_6.predict(X_test))
Epoch 1/150 359/359 - 1s - loss: 0.6328 - recall_6: 0.4692 - val_loss: 0.6663 - val_recall_6: 0.5824 - 1s/epoch - 4ms/step Epoch 2/150 359/359 - 1s - loss: 0.5672 - recall_6: 0.6158 - val_loss: 0.6616 - val_recall_6: 0.5973 - 591ms/epoch - 2ms/step Epoch 3/150 359/359 - 1s - loss: 0.5369 - recall_6: 0.6474 - val_loss: 0.6498 - val_recall_6: 0.6248 - 585ms/epoch - 2ms/step Epoch 4/150 359/359 - 1s - loss: 0.5161 - recall_6: 0.6647 - val_loss: 0.6014 - val_recall_6: 0.6633 - 583ms/epoch - 2ms/step Epoch 5/150 359/359 - 1s - loss: 0.4970 - recall_6: 0.6853 - val_loss: 0.5620 - val_recall_6: 0.7111 - 602ms/epoch - 2ms/step Epoch 6/150 359/359 - 1s - loss: 0.4797 - recall_6: 0.7079 - val_loss: 0.5808 - val_recall_6: 0.7064 - 574ms/epoch - 2ms/step Epoch 7/150 359/359 - 1s - loss: 0.4655 - recall_6: 0.7181 - val_loss: 0.4993 - val_recall_6: 0.7598 - 558ms/epoch - 2ms/step Epoch 8/150 359/359 - 1s - loss: 0.4546 - recall_6: 0.7314 - val_loss: 0.5488 - val_recall_6: 0.7276 - 569ms/epoch - 2ms/step Epoch 9/150 359/359 - 1s - loss: 0.4471 - recall_6: 0.7326 - val_loss: 0.5100 - val_recall_6: 0.7520 - 581ms/epoch - 2ms/step Epoch 10/150 359/359 - 1s - loss: 0.4415 - recall_6: 0.7344 - val_loss: 0.5158 - val_recall_6: 0.7449 - 564ms/epoch - 2ms/step Epoch 11/150 359/359 - 1s - loss: 0.4376 - recall_6: 0.7371 - val_loss: 0.5184 - val_recall_6: 0.7449 - 602ms/epoch - 2ms/step Epoch 12/150 359/359 - 1s - loss: 0.4345 - recall_6: 0.7405 - val_loss: 0.4969 - val_recall_6: 0.7543 - 581ms/epoch - 2ms/step Epoch 13/150 359/359 - 1s - loss: 0.4320 - recall_6: 0.7401 - val_loss: 0.4726 - val_recall_6: 0.7692 - 586ms/epoch - 2ms/step Epoch 14/150 359/359 - 1s - loss: 0.4295 - recall_6: 0.7448 - val_loss: 0.5158 - val_recall_6: 0.7425 - 569ms/epoch - 2ms/step Epoch 15/150 359/359 - 1s - loss: 0.4271 - recall_6: 0.7436 - val_loss: 0.4773 - val_recall_6: 0.7724 - 575ms/epoch - 2ms/step Epoch 16/150 359/359 - 1s - loss: 0.4254 - recall_6: 0.7450 - val_loss: 0.4769 - val_recall_6: 0.7724 - 572ms/epoch - 2ms/step Epoch 17/150 359/359 - 1s - loss: 0.4237 - recall_6: 0.7468 - val_loss: 0.5049 - val_recall_6: 0.7527 - 570ms/epoch - 2ms/step Epoch 18/150 359/359 - 1s - loss: 0.4220 - recall_6: 0.7466 - val_loss: 0.5015 - val_recall_6: 0.7575 - 553ms/epoch - 2ms/step Epoch 19/150 359/359 - 1s - loss: 0.4205 - recall_6: 0.7515 - val_loss: 0.5029 - val_recall_6: 0.7520 - 593ms/epoch - 2ms/step Epoch 20/150 359/359 - 1s - loss: 0.4187 - recall_6: 0.7477 - val_loss: 0.4789 - val_recall_6: 0.7653 - 564ms/epoch - 2ms/step Epoch 21/150 359/359 - 1s - loss: 0.4171 - recall_6: 0.7507 - val_loss: 0.4728 - val_recall_6: 0.7708 - 575ms/epoch - 2ms/step Epoch 22/150 359/359 - 1s - loss: 0.4156 - recall_6: 0.7544 - val_loss: 0.4454 - val_recall_6: 0.7928 - 592ms/epoch - 2ms/step Epoch 23/150 359/359 - 1s - loss: 0.4144 - recall_6: 0.7542 - val_loss: 0.4538 - val_recall_6: 0.7881 - 593ms/epoch - 2ms/step Epoch 24/150 359/359 - 1s - loss: 0.4127 - recall_6: 0.7536 - val_loss: 0.4483 - val_recall_6: 0.7865 - 587ms/epoch - 2ms/step Epoch 25/150 359/359 - 1s - loss: 0.4113 - recall_6: 0.7556 - val_loss: 0.4526 - val_recall_6: 0.7834 - 558ms/epoch - 2ms/step Epoch 26/150 359/359 - 1s - loss: 0.4103 - recall_6: 0.7580 - val_loss: 0.4537 - val_recall_6: 0.7865 - 598ms/epoch - 2ms/step Epoch 27/150 359/359 - 1s - loss: 0.4085 - recall_6: 0.7587 - val_loss: 0.4817 - val_recall_6: 0.7692 - 581ms/epoch - 2ms/step Epoch 28/150 359/359 - 1s - loss: 0.4070 - recall_6: 0.7607 - val_loss: 0.5039 - val_recall_6: 0.7551 - 576ms/epoch - 2ms/step Epoch 29/150 359/359 - 1s - loss: 0.4059 - recall_6: 0.7605 - val_loss: 0.4589 - val_recall_6: 0.7818 - 568ms/epoch - 2ms/step Epoch 30/150 359/359 - 1s - loss: 0.4044 - recall_6: 0.7636 - val_loss: 0.4717 - val_recall_6: 0.7724 - 579ms/epoch - 2ms/step Epoch 31/150 359/359 - 1s - loss: 0.4028 - recall_6: 0.7597 - val_loss: 0.4294 - val_recall_6: 0.7983 - 578ms/epoch - 2ms/step Epoch 32/150 359/359 - 1s - loss: 0.4015 - recall_6: 0.7636 - val_loss: 0.4303 - val_recall_6: 0.7920 - 560ms/epoch - 2ms/step Epoch 33/150 359/359 - 1s - loss: 0.4003 - recall_6: 0.7664 - val_loss: 0.5161 - val_recall_6: 0.7480 - 579ms/epoch - 2ms/step Epoch 34/150 359/359 - 1s - loss: 0.3989 - recall_6: 0.7654 - val_loss: 0.3994 - val_recall_6: 0.8116 - 564ms/epoch - 2ms/step Epoch 35/150 359/359 - 1s - loss: 0.3974 - recall_6: 0.7691 - val_loss: 0.5015 - val_recall_6: 0.7512 - 578ms/epoch - 2ms/step Epoch 36/150 359/359 - 1s - loss: 0.3956 - recall_6: 0.7670 - val_loss: 0.4613 - val_recall_6: 0.7724 - 611ms/epoch - 2ms/step Epoch 37/150 359/359 - 1s - loss: 0.3945 - recall_6: 0.7684 - val_loss: 0.4536 - val_recall_6: 0.7786 - 580ms/epoch - 2ms/step Epoch 38/150 359/359 - 1s - loss: 0.3931 - recall_6: 0.7709 - val_loss: 0.4541 - val_recall_6: 0.7841 - 566ms/epoch - 2ms/step Epoch 39/150 359/359 - 1s - loss: 0.3921 - recall_6: 0.7689 - val_loss: 0.4123 - val_recall_6: 0.8085 - 571ms/epoch - 2ms/step Epoch 40/150 359/359 - 1s - loss: 0.3904 - recall_6: 0.7699 - val_loss: 0.4180 - val_recall_6: 0.8022 - 619ms/epoch - 2ms/step Epoch 41/150 359/359 - 1s - loss: 0.3893 - recall_6: 0.7737 - val_loss: 0.3937 - val_recall_6: 0.8116 - 580ms/epoch - 2ms/step Epoch 42/150 359/359 - 1s - loss: 0.3885 - recall_6: 0.7744 - val_loss: 0.4602 - val_recall_6: 0.7747 - 557ms/epoch - 2ms/step Epoch 43/150 359/359 - 1s - loss: 0.3863 - recall_6: 0.7764 - val_loss: 0.4420 - val_recall_6: 0.7889 - 588ms/epoch - 2ms/step Epoch 44/150 359/359 - 1s - loss: 0.3857 - recall_6: 0.7744 - val_loss: 0.4407 - val_recall_6: 0.7849 - 548ms/epoch - 2ms/step Epoch 45/150 359/359 - 1s - loss: 0.3843 - recall_6: 0.7748 - val_loss: 0.4255 - val_recall_6: 0.7959 - 578ms/epoch - 2ms/step Epoch 46/150 359/359 - 1s - loss: 0.3827 - recall_6: 0.7797 - val_loss: 0.4424 - val_recall_6: 0.7841 - 581ms/epoch - 2ms/step Epoch 47/150 359/359 - 1s - loss: 0.3815 - recall_6: 0.7729 - val_loss: 0.4217 - val_recall_6: 0.7983 - 567ms/epoch - 2ms/step Epoch 48/150 359/359 - 1s - loss: 0.3803 - recall_6: 0.7772 - val_loss: 0.4615 - val_recall_6: 0.7755 - 565ms/epoch - 2ms/step Epoch 49/150 359/359 - 1s - loss: 0.3791 - recall_6: 0.7752 - val_loss: 0.4129 - val_recall_6: 0.8030 - 579ms/epoch - 2ms/step Epoch 50/150 359/359 - 1s - loss: 0.3778 - recall_6: 0.7740 - val_loss: 0.4010 - val_recall_6: 0.8108 - 595ms/epoch - 2ms/step Epoch 51/150 359/359 - 1s - loss: 0.3762 - recall_6: 0.7772 - val_loss: 0.4320 - val_recall_6: 0.7936 - 558ms/epoch - 2ms/step Epoch 52/150 359/359 - 1s - loss: 0.3753 - recall_6: 0.7784 - val_loss: 0.3619 - val_recall_6: 0.8352 - 580ms/epoch - 2ms/step Epoch 53/150 359/359 - 1s - loss: 0.3740 - recall_6: 0.7797 - val_loss: 0.4183 - val_recall_6: 0.7959 - 563ms/epoch - 2ms/step Epoch 54/150 359/359 - 1s - loss: 0.3724 - recall_6: 0.7813 - val_loss: 0.4120 - val_recall_6: 0.8069 - 602ms/epoch - 2ms/step Epoch 55/150 359/359 - 1s - loss: 0.3707 - recall_6: 0.7878 - val_loss: 0.3771 - val_recall_6: 0.8250 - 593ms/epoch - 2ms/step Epoch 56/150 359/359 - 1s - loss: 0.3697 - recall_6: 0.7845 - val_loss: 0.4030 - val_recall_6: 0.8046 - 591ms/epoch - 2ms/step Epoch 57/150 359/359 - 1s - loss: 0.3682 - recall_6: 0.7868 - val_loss: 0.4247 - val_recall_6: 0.7928 - 596ms/epoch - 2ms/step Epoch 58/150 359/359 - 1s - loss: 0.3666 - recall_6: 0.7878 - val_loss: 0.4123 - val_recall_6: 0.7991 - 575ms/epoch - 2ms/step Epoch 59/150 359/359 - 1s - loss: 0.3654 - recall_6: 0.7852 - val_loss: 0.4096 - val_recall_6: 0.7983 - 575ms/epoch - 2ms/step Epoch 60/150 359/359 - 1s - loss: 0.3640 - recall_6: 0.7866 - val_loss: 0.3858 - val_recall_6: 0.8203 - 583ms/epoch - 2ms/step Epoch 61/150 359/359 - 1s - loss: 0.3630 - recall_6: 0.7923 - val_loss: 0.3921 - val_recall_6: 0.8179 - 591ms/epoch - 2ms/step Epoch 62/150 359/359 - 1s - loss: 0.3613 - recall_6: 0.7939 - val_loss: 0.4617 - val_recall_6: 0.7637 - 588ms/epoch - 2ms/step Epoch 63/150 359/359 - 1s - loss: 0.3602 - recall_6: 0.7892 - val_loss: 0.3579 - val_recall_6: 0.8336 - 575ms/epoch - 2ms/step Epoch 64/150 359/359 - 1s - loss: 0.3587 - recall_6: 0.7986 - val_loss: 0.3625 - val_recall_6: 0.8265 - 591ms/epoch - 2ms/step Epoch 65/150 359/359 - 1s - loss: 0.3571 - recall_6: 0.7913 - val_loss: 0.2995 - val_recall_6: 0.8791 - 574ms/epoch - 2ms/step Epoch 66/150 359/359 - 1s - loss: 0.3558 - recall_6: 0.7982 - val_loss: 0.3248 - val_recall_6: 0.8540 - 593ms/epoch - 2ms/step Epoch 67/150 359/359 - 1s - loss: 0.3538 - recall_6: 0.7976 - val_loss: 0.3621 - val_recall_6: 0.8273 - 557ms/epoch - 2ms/step Epoch 68/150 359/359 - 1s - loss: 0.3528 - recall_6: 0.7980 - val_loss: 0.3668 - val_recall_6: 0.8336 - 601ms/epoch - 2ms/step Epoch 69/150 359/359 - 1s - loss: 0.3515 - recall_6: 0.7974 - val_loss: 0.4046 - val_recall_6: 0.8069 - 592ms/epoch - 2ms/step Epoch 70/150 359/359 - 1s - loss: 0.3498 - recall_6: 0.7988 - val_loss: 0.3604 - val_recall_6: 0.8320 - 613ms/epoch - 2ms/step Epoch 71/150 359/359 - 1s - loss: 0.3489 - recall_6: 0.8021 - val_loss: 0.3983 - val_recall_6: 0.8069 - 569ms/epoch - 2ms/step Epoch 72/150 359/359 - 1s - loss: 0.3477 - recall_6: 0.8053 - val_loss: 0.4168 - val_recall_6: 0.7951 - 583ms/epoch - 2ms/step Epoch 73/150 359/359 - 1s - loss: 0.3452 - recall_6: 0.8013 - val_loss: 0.3126 - val_recall_6: 0.8634 - 616ms/epoch - 2ms/step Epoch 74/150 359/359 - 1s - loss: 0.3450 - recall_6: 0.8060 - val_loss: 0.3618 - val_recall_6: 0.8352 - 587ms/epoch - 2ms/step Epoch 75/150 359/359 - 1s - loss: 0.3430 - recall_6: 0.8066 - val_loss: 0.3364 - val_recall_6: 0.8532 - 590ms/epoch - 2ms/step Epoch 76/150 359/359 - 1s - loss: 0.3413 - recall_6: 0.8113 - val_loss: 0.3435 - val_recall_6: 0.8438 - 598ms/epoch - 2ms/step Epoch 77/150 359/359 - 1s - loss: 0.3398 - recall_6: 0.8119 - val_loss: 0.4148 - val_recall_6: 0.7912 - 582ms/epoch - 2ms/step Epoch 78/150 359/359 - 1s - loss: 0.3382 - recall_6: 0.8102 - val_loss: 0.2857 - val_recall_6: 0.8823 - 600ms/epoch - 2ms/step Epoch 79/150 359/359 - 1s - loss: 0.3376 - recall_6: 0.8131 - val_loss: 0.4387 - val_recall_6: 0.7881 - 569ms/epoch - 2ms/step Epoch 80/150 359/359 - 1s - loss: 0.3356 - recall_6: 0.8153 - val_loss: 0.3592 - val_recall_6: 0.8312 - 604ms/epoch - 2ms/step Epoch 81/150 359/359 - 1s - loss: 0.3352 - recall_6: 0.8145 - val_loss: 0.3876 - val_recall_6: 0.8171 - 575ms/epoch - 2ms/step Epoch 82/150 359/359 - 1s - loss: 0.3325 - recall_6: 0.8131 - val_loss: 0.3689 - val_recall_6: 0.8336 - 577ms/epoch - 2ms/step Epoch 83/150 359/359 - 1s - loss: 0.3318 - recall_6: 0.8174 - val_loss: 0.3659 - val_recall_6: 0.8312 - 588ms/epoch - 2ms/step Epoch 84/150 359/359 - 1s - loss: 0.3299 - recall_6: 0.8208 - val_loss: 0.3453 - val_recall_6: 0.8446 - 591ms/epoch - 2ms/step Epoch 85/150 359/359 - 1s - loss: 0.3283 - recall_6: 0.8204 - val_loss: 0.3006 - val_recall_6: 0.8721 - 590ms/epoch - 2ms/step Epoch 86/150 359/359 - 1s - loss: 0.3272 - recall_6: 0.8235 - val_loss: 0.3176 - val_recall_6: 0.8619 - 587ms/epoch - 2ms/step Epoch 87/150 359/359 - 1s - loss: 0.3249 - recall_6: 0.8225 - val_loss: 0.3720 - val_recall_6: 0.8218 - 598ms/epoch - 2ms/step Epoch 88/150 359/359 - 1s - loss: 0.3236 - recall_6: 0.8231 - val_loss: 0.3012 - val_recall_6: 0.8689 - 591ms/epoch - 2ms/step Epoch 89/150 359/359 - 1s - loss: 0.3234 - recall_6: 0.8302 - val_loss: 0.2502 - val_recall_6: 0.8964 - 574ms/epoch - 2ms/step Epoch 90/150 359/359 - 1s - loss: 0.3208 - recall_6: 0.8288 - val_loss: 0.2831 - val_recall_6: 0.8807 - 597ms/epoch - 2ms/step Epoch 91/150 359/359 - 1s - loss: 0.3195 - recall_6: 0.8280 - val_loss: 0.3359 - val_recall_6: 0.8469 - 568ms/epoch - 2ms/step Epoch 92/150 359/359 - 1s - loss: 0.3175 - recall_6: 0.8294 - val_loss: 0.3114 - val_recall_6: 0.8579 - 595ms/epoch - 2ms/step Epoch 93/150 359/359 - 1s - loss: 0.3159 - recall_6: 0.8337 - val_loss: 0.3074 - val_recall_6: 0.8571 - 610ms/epoch - 2ms/step Epoch 94/150 359/359 - 1s - loss: 0.3152 - recall_6: 0.8265 - val_loss: 0.2341 - val_recall_6: 0.9074 - 596ms/epoch - 2ms/step Epoch 95/150 359/359 - 1s - loss: 0.3150 - recall_6: 0.8353 - val_loss: 0.2669 - val_recall_6: 0.8987 - 597ms/epoch - 2ms/step Epoch 96/150 359/359 - 1s - loss: 0.3118 - recall_6: 0.8363 - val_loss: 0.3766 - val_recall_6: 0.8155 - 587ms/epoch - 2ms/step Epoch 97/150 359/359 - 1s - loss: 0.3111 - recall_6: 0.8388 - val_loss: 0.3819 - val_recall_6: 0.8218 - 607ms/epoch - 2ms/step Epoch 98/150 359/359 - 1s - loss: 0.3084 - recall_6: 0.8400 - val_loss: 0.4267 - val_recall_6: 0.7951 - 580ms/epoch - 2ms/step Epoch 99/150 359/359 - 1s - loss: 0.3079 - recall_6: 0.8327 - val_loss: 0.4533 - val_recall_6: 0.7771 - 570ms/epoch - 2ms/step Epoch 100/150 359/359 - 1s - loss: 0.3071 - recall_6: 0.8351 - val_loss: 0.3585 - val_recall_6: 0.8352 - 600ms/epoch - 2ms/step Epoch 101/150 359/359 - 1s - loss: 0.3047 - recall_6: 0.8396 - val_loss: 0.4278 - val_recall_6: 0.7951 - 587ms/epoch - 2ms/step Epoch 102/150 359/359 - 1s - loss: 0.3031 - recall_6: 0.8459 - val_loss: 0.2926 - val_recall_6: 0.8752 - 589ms/epoch - 2ms/step Epoch 103/150 359/359 - 1s - loss: 0.3016 - recall_6: 0.8414 - val_loss: 0.3256 - val_recall_6: 0.8485 - 590ms/epoch - 2ms/step Epoch 104/150 359/359 - 1s - loss: 0.3009 - recall_6: 0.8469 - val_loss: 0.4006 - val_recall_6: 0.8093 - 604ms/epoch - 2ms/step Epoch 105/150 359/359 - 1s - loss: 0.2995 - recall_6: 0.8441 - val_loss: 0.3468 - val_recall_6: 0.8438 - 596ms/epoch - 2ms/step Epoch 106/150 359/359 - 1s - loss: 0.2972 - recall_6: 0.8492 - val_loss: 0.4288 - val_recall_6: 0.7889 - 568ms/epoch - 2ms/step Epoch 107/150 359/359 - 1s - loss: 0.2973 - recall_6: 0.8508 - val_loss: 0.3056 - val_recall_6: 0.8728 - 613ms/epoch - 2ms/step Epoch 108/150 359/359 - 1s - loss: 0.2959 - recall_6: 0.8483 - val_loss: 0.4923 - val_recall_6: 0.7465 - 565ms/epoch - 2ms/step Epoch 109/150 359/359 - 1s - loss: 0.2945 - recall_6: 0.8508 - val_loss: 0.3417 - val_recall_6: 0.8485 - 627ms/epoch - 2ms/step Epoch 110/150 359/359 - 1s - loss: 0.2927 - recall_6: 0.8508 - val_loss: 0.2541 - val_recall_6: 0.8940 - 577ms/epoch - 2ms/step Epoch 111/150 359/359 - 1s - loss: 0.2925 - recall_6: 0.8498 - val_loss: 0.3134 - val_recall_6: 0.8603 - 618ms/epoch - 2ms/step Epoch 112/150 359/359 - 1s - loss: 0.2894 - recall_6: 0.8565 - val_loss: 0.2345 - val_recall_6: 0.9105 - 570ms/epoch - 2ms/step Epoch 113/150 359/359 - 1s - loss: 0.2883 - recall_6: 0.8534 - val_loss: 0.4169 - val_recall_6: 0.8093 - 582ms/epoch - 2ms/step Epoch 114/150 359/359 - 1s - loss: 0.2866 - recall_6: 0.8547 - val_loss: 0.3714 - val_recall_6: 0.8359 - 587ms/epoch - 2ms/step Epoch 115/150 359/359 - 1s - loss: 0.2871 - recall_6: 0.8536 - val_loss: 0.3546 - val_recall_6: 0.8422 - 557ms/epoch - 2ms/step Epoch 116/150 359/359 - 1s - loss: 0.2851 - recall_6: 0.8571 - val_loss: 0.3393 - val_recall_6: 0.8438 - 589ms/epoch - 2ms/step Epoch 117/150 359/359 - 1s - loss: 0.2825 - recall_6: 0.8590 - val_loss: 0.2926 - val_recall_6: 0.8697 - 566ms/epoch - 2ms/step Epoch 118/150 359/359 - 1s - loss: 0.2825 - recall_6: 0.8589 - val_loss: 0.4937 - val_recall_6: 0.7504 - 566ms/epoch - 2ms/step Epoch 119/150 359/359 - 1s - loss: 0.2814 - recall_6: 0.8587 - val_loss: 0.3009 - val_recall_6: 0.8697 - 581ms/epoch - 2ms/step Epoch 120/150 359/359 - 1s - loss: 0.2798 - recall_6: 0.8626 - val_loss: 0.1867 - val_recall_6: 0.9184 - 591ms/epoch - 2ms/step Epoch 121/150 359/359 - 1s - loss: 0.2783 - recall_6: 0.8642 - val_loss: 0.4176 - val_recall_6: 0.8108 - 582ms/epoch - 2ms/step Epoch 122/150 359/359 - 1s - loss: 0.2788 - recall_6: 0.8600 - val_loss: 0.3870 - val_recall_6: 0.8116 - 565ms/epoch - 2ms/step Epoch 123/150 359/359 - 1s - loss: 0.2778 - recall_6: 0.8640 - val_loss: 0.3303 - val_recall_6: 0.8469 - 588ms/epoch - 2ms/step Epoch 124/150 359/359 - 1s - loss: 0.2749 - recall_6: 0.8647 - val_loss: 0.5781 - val_recall_6: 0.7017 - 580ms/epoch - 2ms/step Epoch 125/150 359/359 - 1s - loss: 0.2733 - recall_6: 0.8655 - val_loss: 0.3401 - val_recall_6: 0.8485 - 587ms/epoch - 2ms/step Epoch 126/150 359/359 - 1s - loss: 0.2731 - recall_6: 0.8642 - val_loss: 0.2850 - val_recall_6: 0.8783 - 599ms/epoch - 2ms/step Epoch 127/150 359/359 - 1s - loss: 0.2703 - recall_6: 0.8685 - val_loss: 0.3265 - val_recall_6: 0.8548 - 567ms/epoch - 2ms/step Epoch 128/150 359/359 - 1s - loss: 0.2691 - recall_6: 0.8722 - val_loss: 0.3078 - val_recall_6: 0.8666 - 586ms/epoch - 2ms/step Epoch 129/150 359/359 - 1s - loss: 0.2685 - recall_6: 0.8685 - val_loss: 0.2134 - val_recall_6: 0.9199 - 563ms/epoch - 2ms/step Epoch 130/150 359/359 - 1s - loss: 0.2676 - recall_6: 0.8693 - val_loss: 0.4146 - val_recall_6: 0.7967 - 582ms/epoch - 2ms/step Epoch 131/150 359/359 - 1s - loss: 0.2667 - recall_6: 0.8732 - val_loss: 0.2342 - val_recall_6: 0.9137 - 575ms/epoch - 2ms/step Epoch 132/150 359/359 - 1s - loss: 0.2651 - recall_6: 0.8732 - val_loss: 0.3472 - val_recall_6: 0.8422 - 584ms/epoch - 2ms/step Epoch 133/150 359/359 - 1s - loss: 0.2644 - recall_6: 0.8742 - val_loss: 0.3875 - val_recall_6: 0.8171 - 577ms/epoch - 2ms/step Epoch 134/150 359/359 - 1s - loss: 0.2637 - recall_6: 0.8732 - val_loss: 0.2387 - val_recall_6: 0.8995 - 572ms/epoch - 2ms/step Epoch 135/150 359/359 - 1s - loss: 0.2612 - recall_6: 0.8750 - val_loss: 0.3132 - val_recall_6: 0.8603 - 587ms/epoch - 2ms/step Epoch 136/150 359/359 - 1s - loss: 0.2608 - recall_6: 0.8751 - val_loss: 0.3652 - val_recall_6: 0.8367 - 573ms/epoch - 2ms/step Epoch 137/150 359/359 - 1s - loss: 0.2603 - recall_6: 0.8761 - val_loss: 0.3460 - val_recall_6: 0.8438 - 596ms/epoch - 2ms/step Epoch 138/150 359/359 - 1s - loss: 0.2591 - recall_6: 0.8771 - val_loss: 0.3256 - val_recall_6: 0.8532 - 576ms/epoch - 2ms/step Epoch 139/150 359/359 - 1s - loss: 0.2591 - recall_6: 0.8803 - val_loss: 0.4042 - val_recall_6: 0.8093 - 592ms/epoch - 2ms/step Epoch 140/150 359/359 - 1s - loss: 0.2571 - recall_6: 0.8775 - val_loss: 0.4010 - val_recall_6: 0.8116 - 585ms/epoch - 2ms/step Epoch 141/150 359/359 - 1s - loss: 0.2563 - recall_6: 0.8769 - val_loss: 0.1746 - val_recall_6: 0.9396 - 583ms/epoch - 2ms/step Epoch 142/150 359/359 - 1s - loss: 0.2561 - recall_6: 0.8799 - val_loss: 0.3912 - val_recall_6: 0.8155 - 592ms/epoch - 2ms/step Epoch 143/150 359/359 - 1s - loss: 0.2542 - recall_6: 0.8816 - val_loss: 0.1782 - val_recall_6: 0.9349 - 564ms/epoch - 2ms/step Epoch 144/150 359/359 - 1s - loss: 0.2548 - recall_6: 0.8777 - val_loss: 0.2179 - val_recall_6: 0.9089 - 571ms/epoch - 2ms/step Epoch 145/150 359/359 - 1s - loss: 0.2524 - recall_6: 0.8826 - val_loss: 0.2194 - val_recall_6: 0.9152 - 561ms/epoch - 2ms/step Epoch 146/150 359/359 - 1s - loss: 0.2508 - recall_6: 0.8812 - val_loss: 0.3289 - val_recall_6: 0.8509 - 585ms/epoch - 2ms/step Epoch 147/150 359/359 - 1s - loss: 0.2505 - recall_6: 0.8828 - val_loss: 0.2668 - val_recall_6: 0.8854 - 589ms/epoch - 2ms/step Epoch 148/150 359/359 - 1s - loss: 0.2494 - recall_6: 0.8832 - val_loss: 0.2152 - val_recall_6: 0.9113 - 574ms/epoch - 2ms/step Epoch 149/150 359/359 - 1s - loss: 0.2493 - recall_6: 0.8857 - val_loss: 0.3156 - val_recall_6: 0.8532 - 576ms/epoch - 2ms/step Epoch 150/150 359/359 - 1s - loss: 0.2491 - recall_6: 0.8877 - val_loss: 0.4076 - val_recall_6: 0.8053 - 567ms/epoch - 2ms/step 63/63 [==============================] - 0s 2ms/step - loss: 0.4930 - recall_6: 0.4988
plt.figure(figsize = (15, 8))
plt.plot(history_6.history['loss'])
plt.plot(history_6.history['val_loss'])
plt.title('loss vs Epochs')
plt.ylabel('loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc = 'lower right')
plt.show()
print(classification_report(y_test, test_pred))
cm = confusion_matrix(y_test, test_pred)
labels = np.asarray(
[
[
"{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())
] # flatten will reshape
for item in cm.flatten()
]
).reshape(2, 2)
plt.figure(figsize = (8, 5))
sns.heatmap(cm, annot = labels, fmt = '', xticklabels = ['Not Exited', 'Exited'], yticklabels = ['Not Exited', 'Exited'])
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()
precision recall f1-score support
0 0.88 0.90 0.89 1595
1 0.56 0.50 0.53 405
accuracy 0.82 2000
macro avg 0.72 0.70 0.71 2000
weighted avg 0.81 0.82 0.82 2000
In this case study,